Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
>that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
I will go meta into what you posted here: That people are classifying themselves as "AI skeptics". Many people are treating this in terms of tribal conflict and identity politics. On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits. If we do talk about it as a debate, we can do it when with open minds, and intellectual honesty.
I think much of this may be a reaction to the hype promoted by tech CEOs and media outlets. People are seeing through their lies and exaggerations, and taking positions like "AI/LLMs have no values or uses", then using every argument they hear as a reason why it is bad in a broad sense. For example: Energy and water concerns. That's my best guess about the concern you're braced against.
> I will go meta into what you posted here: That people are classifying themselves as "AI skeptics"
The comment you're replying to is calling other people AI skeptics.
Your advice has some fine parts to it (and simonw's comment is innocuous in its use of the term), but if we're really going meta, you seem to be engaging in the tribal conflict you're decrying by lecturing an imaginary person rather than the actual context of what you're responding to.
To me, "Tip for AI skeptics" reads as shorthand for "Tip for those of you who classify as AI skeptics".
That is why the meta commentary about identity politics made complete sense to me. It's simply observing that this discussion (like so many others) tends to go this way, and suggests a better alternative - without a straw man.
Expecting a purely technical discussion is unrealistic because many people have significant vested interests. This includes not only those with financial stakes in AI stocks but also a large number of professionals in roles that could be transformed or replaced by this technology. For these groups, the discussion is inherently political, not just technical.
I don't really mind if people advocate for their value judgements, but the total disregard for good faith arguments and facts is really out of control. The number of people who care at all about finding the best position through debate and are willing to adjust their position is really shockingly small across almost every issue.
Totally agree. It seems like a symptom of a larger issue: people are becoming increasingly selfish and entrenched in their own bubbles. It’s hard to see a path back to sanity from here.
yeah maybe around the time of Archimedes it was closer to the top, but societies in which people are willing to die for abstract ideas tend to be one... where the value of life isn't quite as high as it is nowadays (ie no matter how much my inner nerd has a love and fascination for that time period, no way i'm pressing the button on any one-way time machines...).
I mean, Archimedes stands out because he searched for the truth and documented it. I'm sure most people on the planet at that time would have burned you for being a witch, or whatever fabled creature was in vogue at the time.
Only among the people who are yelling, perhaps? I find the majority of people I talk with have open minds and acknowledge the opinions of others without accepting them as fact.
> a large number of professionals in roles that could be transformed or replaced by this technology.
Right, "It is difficult get a man to understand something when his salary depends on his not understanding it."
I see this sort of irrationality around AI at my workplace, with the owners constantly droning on about "we must use AI everywhere." They are completely and irrationally paranoid that the business will fail or get outpaced by a competitor if we are not "using AI." Keep in mind this is a small 300 employee, non-tech company with no real local competitors.
Asking for clarification or what they mean by "use AI" they have no answers, just "other companies are going to use AI, and we need to use AI or we will fall behind."
There's no strategy or technical merit here, no pre-defined use case people have in mind. Purely driven by hype. We do in fact use AI. I do, the office workers use it daily, but the reality is it has had no outward/visible effect on profitability, so it doesn't show up on the P&L at the end of the quarter except as an expense, and so the hype and mandate continues. The only thing that matters is appearing to "use AI" until the magic box makes the line go up.
> IMO the move is drop the politics, and discuss things on their technical merits.
I'd love this but it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
It's not unlike when religious people condemn a book they refuse to read. The merits of the book don't matter, it's symbolic opposition to something broader.
No, we've used it, you are creating a strawman argument assuming "AI skeptics" are illiterate and/or incapable of understanding. You ironically are the one refusing to accept the possibility that you are wrong.
Okay, but a lot of people are calling environmental and content theft arguments "political" in an attempt to make it sound frivolous.
It's fine if you think every non-technical criticism against AI is overblown. I use LLMs, but it's perfectly fine to start from a place of whether it's ethical, or even a net good, to use these in the first place.
People saying "ignoring all of those arguments, let's just look at the tech" are, generously, either naive or shilling. Why would we only revisit these very important topics, which are the heart of how the tech would alter our society, after it's been fully embraced?
Well they're separate issues. Someone could plausibly take the position that air travel should be banned for environmental reasons, but that has no relevance to the utility of air travel. If a group of people were loudly proclaiming that planes were not only bad but useless, anyone who routinely uses planes would obviously find them non-credible.
> The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
I keep seeing arguments like this. They sound like a bit like a form of nihilsm. Do you really think we shouldn't worry about risks to the environment simply because we're all hypocrites on that front in one way or another? I get the frustration and have been guilty of using this type of argument myself in the past, but refusing to discuss a problem because the people raising the concern are imperfect human beings doesn't seem like a tenable position.
I think that form of argument is called "whataboutism". Whether flights waste energy or are environmentally unfriendly is really a separate issue. Both things can be bad.
Driving a massive truck in the city is stupid too and most short flights should be replaced with high speed rail. And AI wastes a monumental amount of resources.
I wouldn't ignore those arguments but most of the time, they're so poorly formed (eg. using data without logic), they aren't really worth listening to. If you believe AI provides no value, then any environmental cost is too high for you but you can't convey that by trying to dramatize how high it is. That's dishonest and I think people rightly turn off it.
Right now Open"AI", Oracle and everyone else are burning billions of dollars to buy and run these llms, they raise the price of energy around them, they provide negative economic benefit. It's dishonest of you to pretend that isn't the case.
Politics is the set of activities that are associated with making decisions in groups, or other forms of power relations among individuals, such as the distribution of status or resources.
Most municipalities literally do not have enough spare power to service this 1.4 trillion dollar capital rollout as planned on paper. Even if they did, the concurrent inflation of energy costs is about as political as a topic can get.
Economic uncertainty (firings, wage depression) brought on by the promises of AI is about as political as it gets. There's no 'pure world' of 'engineering only' concerns when the primary goals of many of these billionaires is leverage this hype, real and imagined, into reshaping the global economy in their preferred form.
The only people that get to be 'apolitical' are those that have already benefitted the most from the status quo. It's a privilege.
Hear hear, It's funny having seen the same issue pop up in video game forums/communities. People complaining about politics in their video games after decades of completely straight faced US military propaganda from games like Call of Duty but because they agree with it it wasn't politics. To so many people politics begins where they start to disagree.
There are politics and there are Politics, and I don't think the two of you are using the same definition. 'Making decisions in groups' does not require 'oversimplifying issues for the sake of tribal cohesion or loyalty'. It is a distressingly common occurrence that complex problems are oversimplified because political effectiveness requires appealing to a broader audience.
We'd all be better off if more people withheld judgement while actually engaging with the nuances of a political topic instead of pushing for their team. The capacity to do that may be a privilege but it's a privilege worth earning and celebrating.
My definition is the definition. You cannot nuance wash the material conditions that are increasing tribal polarization. Rising inequality and uncertainty create fear and discontent, people that offer easy targets for that resentment will have more sway.
The rise of populist polemic as the most effective means for driving public behavior is also downstream from 'neutral technical solutions' designed to 'maximize engagement (anger) to maximize profit'. This is not actually a morally neutral choice and we're all dealing with the consequence. Adding AI is fuel for the fire.
I mean, it is intellectually honest to point out that the AI debate at the point is much more a religious or political than strictly technical really. Especially the way tech CEOs hype this as the end of everything.
> On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits.
Zero obligation to satisfy HN audience; tiny proportion of the populace. But for giggles...
Technical merits: there are none. Look at Karpathy's GPT on Github. Just some boring old statistics. These technologies are built on top of mathematical principles in textbooks printed 70-80 years ago.
The sharding and distribution of work across numerous machines is also a well trodden technical field.
There is no net new discovery.
This is 100% a political ploy on the part of tech CEOs who take advantage of the innumerate/non-technical political class that holds power. That class is bought into the idea that massive leverage over resource markets is a win for them, and they won't be alive to pay the price of the environmental destruction.
It's not "energy and water" concerns, it's survival of the species concerns obfuscated by socio-political obligations to keep calm carry on and debate endlessly, as vain circumlocution is the hallmark of the elders whose education was modeled on people being VHS cassettes of spoken tradition, industrial and political roles.
IMO there is little technical merit to most software. Maps, communication. That's all that's really needed. ZIRP era insanity juiced the field and created a bunch of self-aggrandizing coder bros whose technical achievements are copy-paste old ideas into new syntax and semantics, to obfuscate their origins, to get funded, sell books, book speaking engagements. There is no removing any of this from politics as political machinations gave rise to the dumbest era of human engineering effort ever.
The only AI that has merit is robotics. Taking manual labor of people that are otherwise exploited by bougie first worlders in their office jobs. People who have, again with the help of politicians, externalized their biologies real needs on the bodies of poorer illiterates they don't have to see as the first-world successfully subjugated them and moved operations out of our own backyard.
Source: was in the room 30 years ago, providing feedback to leadership how to wind down local manufacturing and move it all over to China. Powerful political forces did not like the idea of Americans having the skills and knowledge to build computers. It ran afoul of their goals to subjugate and manipulate through financial engineering.
Americans have been intentionally screwed out of learning hands on skills with which they would have political leverage over the status quo.
There is no removing politics from this. The situation we are in now was 100% crafted by politics.
Yep. Same for the other direction: there is a very strong correlation between identity politics and praising AI on Twitter.
Then there's us who are mildly disappointed on the agents and how they don't live their promise, and the tech CEOs destroying the economy and our savings. Still using the agents for things that work better, but being burned out for spending days of our time fixing the issues the they created to our code.
The adoption and use of technology frequently (even typically) has a political axis, it's kind of just this weird world of consumer tech/personal computers that's nominally "apolitical" because it's instead aligned to the axis of taste/self-identity so it'll generate more economic activity.
Hey Simon, author here (and reader of your blog!).
I used to share your view, but what changed my mind was reading Hao's book. I don't have it to hand, but if my memory serves, she writes about a community in Chile opposing Google building a data centre in their city. The city already suffers from drought, and the data centre, acccording to Google's own assessment, would abstract ~169 litres of water a second from local supplies - about the same as the entire city's consumption.
If I also remember correctly, Hao also reported on another town where salt water was being added to municipal drinking water because the drought, exacerbated by local data centres, was so severe.
It is indeed hard to imagine these quantities of water but for me, anything on the order of a town or city's consumption is a lot. Coupled with droughts, it's a problem, in my view.
The fact that certain specific data centres are being proposed or built in areas with water issues may be bad, but it does not imply that all AI data centres are water guzzling drain holes that are killing Earth, which is the point you were (semi-implicitly) making in the article.
Just because it doesn’t leave the cycle doesn’t mean it’s not an issue. Where it comes back down matters and as climate change makes wet places wetter and dry places drier, that means it’s less distributed
That said, the water issue is overblown. Most of the water calculation comes from power generation (which uses a ton) and is non-potable water.
The potable water consumed is not zero, but it’s like 15% or something
The big issue is power and the fact that most of it comes from fossil fuels
>The big issue is power and the fact that most of it comes from fossil fuels
This has almost zero to do with the data centers themselves and instead the politicians we vote in.
Simply put we need more 'clean' power generation any way you go about it. Economic growth and production is based on ones ability to produce power. We've been coasting on increasing efficiency for a long time, but we've been in need of larger sources of power and distribution for decades, data centers or not.
With that said DC's shouldn't be built in west texas where it's dry. East of the Mississippi gets enough rain that you build a reservoir and you'll have more than enough water to feed the DC for decades.
The way they measure water consumption is genuinely unbelievably misleading at best. For example measuring the water evaporated from a dams basin if any hydroelectric power is used.
Counting water is genuinely just asinine double counting ridiculousness that makes down stream things look completely insane. Like making a pound of beef look like it consumes 10,000L of water.
In reality of course running your shower for 10 to 15 hours is no where near somehow equivalent to eating beef lasagna for dinner and we would actually have a crisis if people started applying any optimization pressure on these useless metrics.
"I took all your money and gambled it away, but don't worry because it wasn't destroyed, it's still circulating in the economy and will come back someday."
A reduction in what can be used from the summer snowmelt is a problem, regardless of whether equivalent atoms are redeposited in the winter snowfall.
IIRC Google chose to pull out altogether to punish the locals for standing up to them— even though they happily built air-cooled data centers elsewhere.
I mean yes, almost all corporations that have a choice do this. Walmart is one of the better known ones that will put a store right on the edge of a municipality that doesn't want one and cause all kinds of issues for the city at hand.
I'm conflicted. Zooming out, the problem isn't with AI specifically but economic development in general. Everything has a side effect.
For decades we've been told we shouldn't develop urban centers because of how it development affects local communities, but really it just benefited another class of elites (anonymous foreign investors), and now housing prices are impoverishing younger generations and driving homelessness.
Obviously that's not a perfect comparison to AI, which isn't as necessary, but I think the anti-growth argument isn't a good one. Democracies need to keep growing or authoritarian states will take over who don't care so much about human rights. (Or, authoritarian governments will take over democracies.)
There needs to be a political movement that's both pro-growth and pro-humanity, that is capable of making hard or disruptive decisions that actually benefits the poor. Maybe that's a fantasy, but again, I think we should find ways to grow sustainably.
Nestle is and has been 10000x worse for global water security than all other companies and countries combined because nobody in the value chain cares about someone else’s aquifer.
It’s a social-economic problem of externalities being ignored , which transcends any narrow technological use case.
What you describe has been true for all exported manufacturing forever.
My interpretation was "If an industry that actively works to harm the global health of humanity through their addictive and unhealthy food products is using way more water and we're OK with it, maybe we should give a pass to the industry using a fraction of that water to improve human productivity."
Ton of nuance in my characterizations of both industries, of course, but to a first approximation they are accurate.
Data center cooling towers have to use fresh rather than salt water, but they don't care about bacterial contamination or toxic traces of arsenic, antimony, and fluorine. Agriculture also has to use fresh rather than salt water. I can't think of any circumstances where water that was usable for agriculture wouldn't also be usable for cooling data centers—except when the farmer owns the water and the data center operator doesn't.
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
Not to mention that this is likely a very short-term debate. Modern AI/ML/compute has been a huge boost to fusion R&D, and tons of AI/tech capital is flowing into companies like Helion and CFS. Necessity is the mother of invention and all.
It may be a valid criticism today, but no one will be complaining about AI's environmental impact after those first few plants go live and mass production begins within the next decade. Knock on wood.
I'm skeptical about that. Helion in particular might work (https://www.youtube.com/watch?v=HlNfP3iywvI) but all the other fusion-energy companies' plans are to convert fusion energy to thermal energy and then convert the thermal energy to electrical energy in the usual way, using steam engines. That's never going to be cheaper than thermal energy powering steam engines, because thermal energy powering steam engines is part of it.
But solar energy is.
Helion's strategy is to directly convert the hot plasma's expansion into electrical energy by, basically, pushing against a powerful magnetic field. This is potentially higher efficiency than steam engines (because it's working at a higher temperature) and potentially higher power density and reliability. So it could at least theoretically get to be cheaper than steam engines. But I think it's more likely to be far more expensive for the next several decades.
Just because there are worse abuses elsewhere doesn't mean datacenters should get a pass.
Golf and datacenters should have to pay for their externalities. And if that means both are uneconomical in arid parts of the country then that's better than bankrupting the public and the environment.
> I asked the farmer if he had noticed any environmental effects from living next to the data centers. The impact on the water supply, he told me, was negligible. "Honestly, we probably use more water than they do," he said. (Training a state-of-the-art A.I. requires less water than is used on a square mile of farmland in a year.) Power is a different story: the farmer said that the local utility was set to hike rates for the third time in three years, with the most recent proposed hike being in the double digits.
The water issue really is a distraction which harms the credibility of people who lean on it. There are plenty of credible reasons to criticize data enters, use those instead!
The other reason water usage is a bad thing to focus on is that datacenters don't inherently have to use water. It's not like servers have a spigot where you pour water in and it gets consumed.
Water is used in modern datacenters for evaporative cooling, and the reason it's used is to save energy -- it's typically around 10% more energy efficient overall than normal air conditioning. These datacenters often have a PUE of under 1.1, meaning they're over 90% efficient at using power for compute, and evaporative cooling is one of the reasons they're able to achieve such high efficiency.
If governments wanted to, they could mandate that datacenters use air conditioning instead of evaporative cooling, and water usage would drop to near zero (just enough for the restrooms, watering the plants, etc). But nobody would ever seriously suggest doing this because it would be using more of a valuable resource (electricity / CO2 emissions) to save a small amount of a cheap and relatively plentiful resource (water).
"...So to wrap up, I misread the Berkeley Report and significantly underestimated US data center water consumption. If you simply take the Berkeley estimates directly, you get around 628 million gallons of water consumption per day for data centers, much higher than the 66-67 million gallons per day I originally stated..."
163.7 billion / 43,070 billion * 100 = 0.38 - less than half a percentage point.
It's very easy to present water numbers in a way that looks bad until you start comparing them thoughtfully.
I think comparing data center water usage to domestic water usage by people living in towns is actually quite misleading. UPDATE: I may be wrong about this, see following comment: https://news.ycombinator.com/item?id=45926469#45927945
The vast majority of water in agriculture goes to satisfy our taste buds, not nourish our bodies. Feed crops like alalfa consume huge amounts of water in the desert southwest but the desert climate makes it a great place to grow and people have an insatiable demand for cattle products.
We could feed the world with far less water consumption if we opted not to eat meat. Instead, we let people make purchasing decisions for themselves. I'm not sure why we should take a different approach when making decisions about compute.
> We could feed the world with far less water consumption if we opted not to eat meat.
If you look at the data for animals, that’s not really true. See [1] especially page 22 but the short of it is that the vast majority of water used for animals is “green water” used for animal feed - that’s rainwater that isn’t captured but goes into the soil. Most of the plants used for animal feed don’t use irrigation agriculture so we’d be saving very little on water consumption if we cut out all animal products [2]. Our water consumption would even get a lot worse because we’d have to replace that protein with tons of irrigated farmland and we’d lose the productivity of essentially all the pastureland that is too marginal to grow anything on (50% of US farmland, 66% globally).
Animal husbandry has been such a successful strategy on a planetary scale because it’s an efficient use of marginal resources no matter how wealthy or industrialized you are. Replacing all those calories with plants that people want to actually eat is going to take more resources, not less, especially when you’re talking about turning pastureland into productive agricultural land.
That paper makes the opposite argument than you thought it made. Even from a freshwater perspective it’s more water-efficient to get calories/protein/fat directly from crops than from animal products.
Since you like their work, the authors of your paper answered that question more generally here https://link.springer.com/article/10.1007/s10021-011-9517-8 where they conclude "The water footprint of any animal product is larger than the water footprint of crop products with equivalent nutritional value".
You make some often debunked claims, like we'd have to plant more crops to feed humans directly if we stopped eating meat.
This shouldn't make intuitive sense to you since animals eat feed grown on good cropland (98% of the water footprint of animal ag) that we could eat directly, and we lose 95% of the calories when we route crops through animals.
I mean it's even simpler. Almonds are entirely non essential (many other more water efficient nuts) to the food supply and in California consume more water than the entire industrial sector, and a bit more than all residential usage (~5 million acre-feet of water).
Add a datacenter tax of 3x to water sold to datacenters and use it to improve water infrastructure all around. Water is absolutely a non-issue medium term, and is only a short term issue because we've forgotten how to modestly grow infrastructure in response to rapid changes in demand.
Growing almonds is just as essential as building an AI. Eating beef at the rate americans do is not essential. Thats where basically all the water usage is going.
Iran's ongoing water crisis is an example. One cause of it is unnecessary water-intensive crops that they could have imported or done without (just consume substitutes).
It's a common reasoning error to bundle up many heterogeneous things into a single label ("agriculture!") and then assign value to the label itself.
I am surprised by your analytical mistake of comparing irrigation water with data-center water usage...
They are not equivalent. Data centers primarily consume potable water, whereas irrigation uses non-potable or agricultural-grade water. Mixing the two leads to misleading conclusions on the impact.
That's a really good point - you're right, comparing data center usage to potable water usage by towns is a different and more valid comparison than comparing with water for irrigation.
They made a good point, but keep in mind that they're doing a "rules for thee, not for me" sometimes.
The same person who mentioned potable water being an important distinction also cited a report on data center water consumption that did not make the distinction (where the 628M number came from).
The factual soundness of my argument is independent of the report quality :-) the report influences comprehension, not correctness...
The fact data centers are already having a major impact on the public water supply systems is known, by the decisions some local governments are forced to do, if you care to investigate...
"...in some regions where data centers are concentrated—and especially in regions already facing shortages—the strain on local water systems can be significant. Bloomberg News reports that about two-thirds of U.S. data centers built since 2022 are in high water-stress areas.
In Newton County, Georgia, some proposed data centers have reportedly requested more water per day than the entire county uses daily. Officials there now face tough choices: reject new projects, require alternative water-efficient cooling systems, invest in costly infrastructure upgrades, or risk imposing water rationing on residents...."
What counts as data center water consumption here? There are many ways to arguably come up with a number.
Does it count water use for cooling only, or does it include use for the infrastructure that keeps it running (power generation, maintenance, staff use, etc.)
Is this water evaporated? Or moved from A to B and raised a few degrees.
This is the real point. Just measuring the amount of water involved makes no sense. Taking 100 liters of water from a river to cool a plant and dumping them back in a river a few degrees warmer is different from taking 100 liters from a fossil acquifer to evaporatively cool the same plant.
A farmer is a valuable perspective but imagine asking a lumberjack about the ecological effects of deforestation, he might know more about it than an average Joe, but there's probably better people to ask for expertise?
> Honestly, we probably use more water than they do
This kind of proves my point, regardless of the actual truth in this regard, it's a terrible argument to make: availability of water starts to become a huge problem in a growing amount of places, and this statement implies the water usage of something, that in basic principle doesn't need water at all, uses comparable amount of water as farming, which strictly relies on water.
I think the point here is that objecting to AI data center water use and not to say, alfalfa farming in Arizona, reads as reactive rather than principled. But more importantly, there are vast, imminent social harms from AI that get crowded out by water use discourse. IMO, the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology.
But if I say "I object to AI because <list of harms> and its water use", why would you assume that I don't also object to alfalfa farming in Arizona?
Similarly, if I say "I object to the genocide in Gaza", would you assume that I don't also object to the Uyghur genocide?
This is nothing but whataboutism.
People are allowed to talk about the bad things AI does without adding a 3-page disclaimer explaining that they understand all the other bad things happening in the world at the same time.
If you take a strong argument and through in an extra weak point, that just makes the whole argument less persuasive (even if that's not rational, it's how people think).
You wouldn't say the "Uyghur genocide is bad because of ... also the disposable plastic crap that those slave factories produce is terrible for the environment."
Plastic waste is bad but it's on such a different level from genocide that it's a terrible argument to make.
Adding a weak argument is a red flag for BS detectors. It's what prosecutors do to hoodwink a jury into stacking charges over a singular underlying crime.
Because your argument is more persuasive to more people if you don't expand your criticism to encompass things that are already normalized. Focus on the unique harms IMO.
I don't think there's a world where a water use tax is levied such that 1) it's enough for datacenters to notice and 2) doesn't immediately bankrupt all golf courses and beef production, because the water use of datacenters is just so much smaller.
We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Beef, I guess, is a popular type of food. I’m under the impression that most of us would be better off eating less meat, maybe we could tax water until beef became a special occasion meal.
I'm saying that if you taxed water enough for datacenters to notice, beef would probably become uneconomical to produce at all. Maybe a good idea! But the reason datacenters would keep operating and beef production wouldn't is that datacenters produce way more utility per gallon.
You can easily write a law that looks like this:
There is now a water usage tax. It applies only to water used for data-centers. It does not apply to residential use, agricultural use, or any other industrial use.
We do preferential pricing and taxing all the time. My home's power rate through the state owned utility is very different than if I consumed the exact same amount of power, but was an industrial site. I just checked and my water rate at home is also different than if I were running a datacenter. So in all actuality we already discriminate for power and water based on end use. at least where I live. Most places I have lived have different commercial and residential rates.
In other words, the price of beef can stay the same.
>In other words, the price of beef can stay the same.
And yet, if you believe the environmentalist argument in the first place, the price of beef should go up for the damages it causes. Hence why a lot of people think the people complaining about AI are wearing an environmentalist mask, rather than having an actual care about the environment.
Pigouvian taxes are fine, but should be applied across all sources of damage.
A lot of beef is produced in such a way that taxing municipal water won't make a material difference. Even buying market rate water rights in the high desert, which already happens in beef production, is a pretty small tariff on the beef.
> We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Might as well get rid of all the lawns and football fields while we’re at it.
Water taxes should probably be regional. The price of water in the arid Southwest is much higher than toward the East coast. You might see both datacenters and beef production moving toward states like Tennessee or Kentucky.
My perspective from someone who wants to understand this new AI landscape in good faith. The water issue isn't the show stopper it's presented as. It's an externality like you discuss.
And in comparison to other water usage, data centers don't match the doomsday narrative presented. I know when I see it now, I mentally discount or stop reading.
Electricity though seems to be real, at least for the area I'm in. I spent some time with ChatGPT last weekend working to model an apples:apples comparison and my area has seen a +48% increase in electric prices from 2023-2025. I modeled a typical 1,000kWh/month usage to see what that looked like in dollar terms and it's an extra $30-40/month.
Is it data centers? Partly yes, straight from the utility co's mouth: "sharply higher demand projections—driven largely by anticipated data center growth"
With FAANG money, that's immaterial. But for those who aren't, that's just one more thing that costs more today than it did yesterday.
Coming full circle, for me being concerned with AI's actual impact on the world, engaging with the facts and understanding them within competing narratives is helpful.
Farmland, AI data centers, and golf courses do not provide the same utility for water used. You are not making an argument against the water usage problem, you are only dismissing it.
Almonds are pretty cherry picked here as notorious for their high water use. Notably, we're not betting an entire economy and pouring increasing resources into almond production, either. Your example would be even more extreme if you chose crops like the corn used to feed cattle. Feeding cows alone requires 21.2 trillion gallons per year in the US.
The people advocating for sustainable usage of natural resources have already been comparing the utility of different types of agriculture for years.
Comparatively, tofu is efficient to produce in terms of land use, greenhouse gas emissions, and water use, and can be made shelf-stable.
People have been sounding the alarm about excessive water diverted to almond farming for many years though, so that doesn't really help the counter-argument.
AI has way more utility than you are claiming and less utility than Sam Altman and the market would like us to believe. It’s okay to have a nuanced take.
I'm more unhappy than happy, as there are plenty of points about the very real bad side of AI that are hurt by such delusional and/or disingenuous arguments.
That is, the topic is not one where I have already picked a side that I'd like to win by any means necessary. It's one where I think there are legitimate tradeoffs, and I want the strongest arguments on both sides to be heard so we get the best possible policies in the end.
Well, I don't like marzipan, so both are useless? Or maybe different people find uses/utility from different things, what is trash for one person can be life saving for another, or just "better than not having it" (like you and Marzipan it seems).
ok in that case you don't need to pick on water in particular, if it has no utility at all then literally any resource use is too much, so why bother insisting that water in particular is a problem? It's pretty energy intensive, eg.
AI has no utility _for you_ because you live in this bubble where you are so rabidly against it you will never allow yourself to acknowledge it has non-zero utility.
What does it mean to “use” water? In agriculture and in data centers my understanding is that water will go back to the sky and then rain down again. It’s not gone, so at most we’re losing the energy cost to process that water.
The problem is that you take the water from the ground, and you let it evaporate, and then it returns to... Well to various places, including the ground, but the deeper you take the water from (drinking water can't be taken from the surface, and for technological reasons drinking water is used too) the more time it takes to replenish the aquifer - up to thousands of years!
Of course surface water availability can also be a serious problem.
No it’s largely the same situation I think. I was drawing a distinction between agricultural use and maybe some more heavy industrial uses while the water is polluted or otherwise rendered permanently unfit for other uses.
Other people might have other preferences. Maybe we could have a price system where people can express their preferences by paying for things with money, providing more money to the product which is in greater demand?
Sure.. Except some people / companies have so much more money, they can demand impractical things and pay above-market rates for them, causing all others to scramble to live day-to-day with the distorted market.
Right, I think a data center produces a heck of a lot more economic and human value in a year - for a lot more people - than the same amount of water used for farming or golf.
The water intensity of American food production would be substantially less if we gave up on frivolous things like beef, which requires water vastly out of proportion to its utility. If the water numbers for datacenters seem scary then the water use numbers for the average American's beef consumption is apocalyptic.
While I agree, the "meat is not sustainable" argument is literal, and evidenced in beef prices rising as beef consumption lowers over the past years. Beef is moving along the spectrum from having had been a "staple" to increasingly being a luxury.
The US never gave up eating lobster either, but many here have never had lobster and almost nobody has lobster even once a week. It's a luxury which used to be a staple.
That depends how sentient a chicken is: their brains are of similar complexity to the larger of these models, counting params as synapses.
Also, while I'm vegetarian going on vegan, welfare arguments are obviously not relevant in response to an assertation that Americans aren't going to give up meat, because if animal welfare was relevant then Americans would give up meat.
> Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds.
Both alfalfa and almonds contain a lot of nutrients you dont find in large enough amounts (or at all) in corn and potatoes though. And alfalfa improves the soil but fixating nitrogen. Sure almonds require large amounts of water. Maybe alfalfa does as well? And of course it depends on if they are grown for human consumption or animal.
Water usage largely depends on the context, if the water source is sustainable, and if it is freshwater.
Of course water used up will eventually evaporate, and produce rainfall in the water cycle, but unfortunately at many places "fossil" water is used up, or more water used in an area then the watershed can sustainably support.
This is a constant source of miscommunication about water usage, and that of agriculture also. It is very different to talk about the water needs to raise a cow in eg. Colorado and in Scotish highlands, but this is usually removed from the picture.
The same context should be considered for datacenters.
Another issue is that you could, in principle, build data centers in places where you don't need to evaporate water to cool them. For example, you could use a closed loop water cooling system and then sink that heat into the air or into a nearby body of water. OVH's datacenter outside Montreal¹ does this, for example. You can also use low carbon energy sources to power the data center (nuclear or hydro are probably the best because their production is reliable and predictable).
Unlike most datacenters, AI datacenters being far away from the user is okay since it takes on the order of seconds to minutes for the code to run and generate a response. So, a few hundred milliseconds of latency is much more tolerable. For this reason, I think that we should pick a small number of ideal locations that have a combination of weather that permits non-sub-ambient cooling and have usable low carbon resources (either hydropower is available and plentiful, or you can build or otherwise access nuclear reactors there), and then put the bulk of this new boom there.
If you pick a place with both population and a cold climate, you could even look into using the data center's waste heat for district heating to get a small new revenue stream and offset some environmental impact.
I think the water use arguments are relevant, particularly in regions of the world and US (CA) where potable water is scarce, but land and electricity are available .
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
One of the biggest problems in environmental education (imo) is the lack of notion regarding the footprint of products and services we consume daily; from the water and CO2 costs of a meal, of a heated pool, of a car, etc. It is eye-opening.
I first came across this type of info with the book "How Bad Are Bananas", from Mike Berners-Lee. I really enjoyed it, and I just saw that the new edition even includes stuff like hosting a World Cup, data centers, and space tourism!
It should give a good foundation to start talking about it.
Water location matters. Is the data center in a desert with scarce potable water for locals? Or is next to a large Canadian lake, plenty of potable water, with people who want to trade something for currency so they can put avocados in their salad?
Some time ago, I read the environmental impact assessment for a proposed natural gas thermal power plant, and in it they emphasized that their water usage was very low (to the point that it fit within the unused part of the water usage allowance for an already existing natural gas thermal power plant on the same site) because they used non-evaporative cooling.
What prevents data centers from using non-evaporative cooling to keep their water usage low? The water usage argument loses a lot of its relevant in that case.
In europe several power plants get shut down each summer because the heated water from those plants would have significant impact on the local wildlife.
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing) 177-comment discussion on a small component of the author's argument. The author's argument is an important one that hardly hinges at all on water usage specifically, given the vast human and financial capital invested in LLM buildout so far.
The problem is more one of scale: a million liters of water is less than half of a single Olympic-sized swimming pool. A single acre of alfalfa typically requires 4.9 - 7.6 million liters a year for irrigation. Also, it's pretty easy to recycle the data center water, since it just has to cool and be sent back, but the irrigation water is lost to transpiration and the recycling-by-weather process.
So, even if there's no recycling, a data center that is said to consume "millions" rather than tens or hundreds of millions is probably using less than 5 acres of alfalfa in consumption, and in absolute terms, this requires only a swimming-pool or two of water per years. It's trivial.
I think the source is the bigger problem. If they take the water from sources which are already scarce, the impact will be harsh. There probably wouldn't be any complaints if they would use sewerage or saltwater from the ocean.
> Also, it's pretty easy to recycle the data center water, since it just has to cool
Cooling and returning the water is not always that simple. I don't know specifically about datacentres, but I know about wasting clean water in other areas, cooling in power plants, industry, etc. and there it can have a significant impact on the cycle. At the end it's a resource which is used at least temporary, which has impact on the whole system.
> If they take the water from sources which are already scarce, the impact will be harsh.
Surprised I had to scroll down this far to see this mentioned.
The water use argument is highly local and depends on where we are building these data centers. Are you building in the great lakes region with plenty of fresh water and no water scarcity issues (yet)? Ok fine.
But we aren't building there. We're building in Arizona, Nevada, Nebraska, Iowa putting further stress in an area that water scarcity is already an issue, or soon going to become one due to long term drought conditions. Or Texas, which already has problems with their power grid.
We're building in these locations because they're cheap. If we're going do to this, we need to not let the bottom line be the sole driving decision of data center locations. If it's not profitable to build elsewhere, don't build it until you've figured out how to make it efficient enough to where it is profitable.
On the water front, in my area agriculture flood irrigates with pristine aquifer water, while surface water gets treated and dumped into the drinking supply. This is due to the economics of historic water rights.
Not really. The majority of data center water withdrawal (total water input) is consumed ("lost" to evaporation etc...) with a minority of it discharged (returned in liquid form). I believe it's on the order of 3/4ths consumed, but that varies a lot by local climate and cooling technology.
There's lots of promising lower-consumption cooling options, but seems like we are not yet seeing that in a large fraction of data centers globally.
Water and power are local issues. And data center use of water and power is already, currently having local impact on politics. I saw ads about it during the election cycle that just concluded. Candidates had to answer questions about it at debates and in interviews.
People are using these arguments for the simple reason that they demonstrably resonate with average people who live near data centers.
They probably don’t resonate with people who have plenty of income and/or do not compete with data centers locally for resources.
The nice thing about the data center water usage panic is that whenever someone appeals to it, I immediately know that either they haven't done their homework or they're arguing in bad faith.
Water can range from serious concern to NBD depending on where the data center is located, where the water is coming from, and the specific details of how the data center's cooling systems are built.
To say that it's never an issue is disingenuous.
Additionally one could image a data center built in a place with a surplus of generating capacity. But in most cases, it has a big impact on the local grid or a big impact on air quality if they bring in a bunch of gas turbines.
We really need to stop tying people's retirement to the market. I've already lost ground due to 2008, and COVID, and while I was young, I lived through my parents suffering through dotcom as well.
It's long past time we have a universal public pension, funded at least in part with a progressive wealth tax, or least go back to private defined benefit pensions to where the employer or pension fund bears the risk rather than the individual.
Supplement it with market speculation if you want, but we need something guaranteed for everyone that's a little more robust and provides a better living than the paltry social security we have now.
The water argument rings a bit hollow for me not due to whataboutism but more that there's an assumption that I know what "using" water means, which I am not sure I do. I suspect many people have even less of an idea than I do so we're all kind of guessing and therefore going to guess in ways favorable to our initial position whatever that is.
Perhaps this is the point, maybe the political math is that more people than not will assume that using water means it's not available for others, or somehow destroyed, or polluted, or whatever. AFAIK they use it for cooling so it's basically thermal pollution which TBH doesn't trigger me the same way that chemical pollution would. I don't want 80c water sterilizing my local ecosystem, but I would guess that warmer, untreated water could still be used for farming and irrigation. Maybe I'm wrong, so if the water angle is a bigger deal than it seems then some education is in order.
If water is just used for cooling, and the output is hotter water, then it's not really "used" at all. Maybe it needs to be cooled to ambient and filtered before someone can use it, but it's still there.
If it was being used for evaporative cooling then the argument would be stronger. But I don't think it is - not least because most data centres don't have massive evaporative cooling towers.
Even then, whether we consider it a bad thing or not depends on the location. If the data centre was located in an area with lots of water, it's not some great loss that it's being evaporated. If it's located in a desert then it obviously is.
If you discharge water into a river, there are environmental limits to the outlet temperature (this is a good thing btw). The water can't be very hot. That means you need to pump a large volume of water through because you can only put a small amount of energy into each kg of water.
If you evaporate the water on the other hand, not only is there no temperature limit but it also absorbs the latent heat of vaporisation. The downside is it's a lot more complex and also the water is truly consumed rather than just warming it up.
Put that way, any electricity usage will have some "water usage" as power plants turn up their output (and the cooling pumps) slightly. And that's not even mentioning hydroelectric plants!
I did some napkin math on data center water usage for a 500MW data center in the Columbia River valley.
It uses as much water per year as 200 acres of alfalfa in California’s Central Valley. There are around 1M acres of alfalfa growing in California.
2.5MW of data center capacity is roughly equal to 1 acre of irrigated alfalfa in water usage. If you’re pulling fossil aquifer water, open loop evaporative cooling may not be the best idea, but there are plenty of places east of 100 degrees west in the US that have virtually ‘unlimited’ water where cooling towers are a great idea since they almost double the COP of a chilled water system.
I went down that “water use” rabbit hole a month ago and basically… it’s just a bunch of circular reporting that was based on some paper published in 2023[1]. For ChatGPT 3.5 they claimed “500ml for 10-50 responses”. In 2024, Washington Post published an article that took their claim and said “519 milliliters per email”[2] but didn’t source that from the original paper… that “shocking headline” took off and got widely circulated and cited directly, treating the WaPo calculation as if it were the original research finding. Then tech blogs and advocacy sites ran with it even harder, citing each other instead of any actual research[3].
If you look at the original paper they are quite upfront with the difficulty of estimating water use. It’s not public data—in fact it’s usually a closely held trade secret, plus it’s got all kinds of other issues like you don’t know where the training happened, when it happened, what the actual cooling efficiency was, etc. The researchers were pretty clear about these limitations in the actual paper.
Basically, it’s urban legend at this point. When OpenAI’s CEO later said ChatGPT uses ~0.3ml per query, that’s roughly 100x less than the viral claims.
"Presented by the USGA" (the United States Golf Association) gave me a wry chuckle there.
That said, here are the relevant numbers from that 2012 article in full:
> Most 18-hole golf facilities utilize surface waters (ponds, lakes) or on-site irrigation wells. Approximately 14 percent of golf facilities use water from a public municipal source and approximately 12 percent use recycled water as a source for irrigation.
> Specific water sources for 18-hole courses as indicated by participants are noted below:
> 52 percent use water from ponds or lakes.
> 46 percent use water from on-site wells.
> 17 percent use water from rivers, streams and
creeks.
> 14 percent use water from municipal water systems.
It's not about it being scary, its about it being a gigantic, stupid waste of water, and for what? So that lazy executives and managers can generate their shitty emails they used to have their comms person write for them, so that students can cheat on their homework, or so degens can generate a video of MLK dancing to rap? Because thats the majority of the common usage at this point and creating the demand for all these datacenters. If it was just for us devs and researchers, you wouldn't need this many.
Whether it's a "gigantic" waste of water depends on what those figures mean. It's very important to understand if 25 million liters of water per year is a gigantic number or not.
For comparison it's about 10 olympic-sized swimming pools worth of water, doesn't seem very significant to me. Unless you're going to tell people they're not allowed swimming pools any more because swimming doesn't produce enough utility?
And at any rate, water doesn't get used up! It evaporates and returns to the sky to rain down again somewhere else, it's the most renewable resource in the entire world.
And also, none of those current use cases are a real benefit to society, outside of maybe research cases.
The only benefit is to the already wealthy owner class that is itching to not have to pay for employees anymore because it impacts their bottom line (payroll is typically the largest expense).
It's not like we are making robots to automate agriculture and manufacturing to move toward a post scarcity, moneyless society, which would have real benefits. No, instead we have AI companies hyping up a product whose purpose (according to them) is so that already wealthy people can hoard more wealth and not have to pay for employees. It's promising to take away a large portion of the only high-paying jobs we have left for the average person without an advanced degree.
Me being able to write software a little faster, without hiring a junior, is a net negative to society rather than a benefit.
You appear to be arguing against using technology to boost human efficiency on a forum full of software engineers who've dedicated their careers to building software that makes humans more efficient.
If we aren't doing that then why are we building software?
Because the stated goal of generative AI is not to make an individual more efficient, it's to replace that individual all together and completely eliminate the bottom rungs of the professional career ladder.
Historically software that made humans more efficient resulted in empowerment for the individual, and also created a need for new skilled roles. Efficiency gains were reinvested into the labor market. More people could enter into higher paying work.
With generative AI, if these companies achieve their stated goals, what happens to the wealth generated by the efficiency?
If we automate agriculture and manufacturing, the gain is distributed as post-scarciaty wealth to everyone.
If we automate the last few remaining white-collar jobs that pay a living wage, the gain is captured entirely by the capital owners & investors via elimination of payroll, while society only loses one of its last high-paying ladders for upward mobility.
Nobody lost their career because we built a faster operating system or a better compiler. With generative AI's stated goals, any efficiency gains are exclusively for those at the very top, while everyone else gets screwed.
Now, I'll concede and say, that's not the AI companies' fault. I'm not saying we shouldn't magically stop developing this technology, but we absolutely need our governments to start thinking about the ramifications it can have and start seriously considering things like UBI to be prepared for when the bottom falls out of the labor market.
I'm not a fan of of the "replace workers with AI" thing myself - I'm much more excited about AI as augmentation for existing workers so they can take on more challenging tasks.
Its not gigantic and its not a waste. Brainrot creates massive economic value that can be used to pay people for products you are more happy to consume.
Arguments in isolation are usually poor. The water usage arguments usually (always?) comes along with a bunch of other arguments, including power consumption, workers rights, consumer protection, public safety, enshittifcation, etc.
When evaluating the economical cost or morality of a thing, (just like when training a machine learning model) the more data you consider the more accurate the result (although just like statistical modelling it is worth to be wary of overfitting).
As for food production; that might be important? IDK, I am not a silicon "intelligence" so what do I know? Also, I have to "eat". Wouldn't it be a wonderful world if we can just replace ourselves, so that agriculture is unnecessary, and we can devote all that water to AGI.
TIL that the true arc of humanity is to replace itself!
Given the difference in water usage, more data centers does not mean less water for agriculture in any meaningful way.
If you genuinely want to save water you should celebrate any time an acre of farm land is converted into an acre of data center - all the more water for the other farms!
Yes, it is worthwhile to ask how much value we get.
And a whole bunch of us are saying we don't see the value in all these datacenters being built and run at full power to do training and inference 24/7, but you just keep ignoring or dismissing that.
It is absolutely possible that generative AI provides some value. That is not the same thing as saying that it provides enough value to justify all of the resources being expended on it.
The fact that the amount of water it uses is a fraction of what is used by agriculture—which is both one of the most important uses humans can put water to, as well as, AIUI, by far the single largest use of water in the world—is not a strong argument that its water usage should be ignored.
it is also a fraction of golf courses which you again ignore. this is just typical "don't do anything!!" ism. there's no argument here.. even if data centres used .00001 millilitre of water you would say the same thing.
Oh, I think golf courses shouldn't exist. They're awful in a number of ways. You want to play golf? VR or minigolf.
But (as I pointed out elsewhere in this discussion [0]) why should I have to mention everything that uses water in a way I think is detrimental in order to be allowed to talk about why I think this thing uses water in a way that is detrimental?
Farms already produce more than enough food to feed everyone (and, indeed, the excess is a feature because food security is really important). The reason not everyone is fed is not due to needing to divide water resources between farms and other uses.
Stop eating beef. With the water saved we can grow enough food for any realistic human population. Ok we solved this one. Or do humans NEED burgers as well? We can already feed all people, any starvation is strictly a political problem not a food existing on the planet problem
Going only by the effective need of humans is a bad argument. A lot of farmers wouldn't survive without subsidies and are not vital to our food supply.
We produce enough food for everyone already, and then waste a huge amount of it. Our food problem isn't about producing more, it's about distributing what we have.
Eventually people stop building more data centers as food becomes scarce and expensive, and farms become the hot new thing for the stock market, cereal entrepreneurs become the new celebrities and so on. Elon Husk, cereal magnate.
I agree that those numbers can seem huge without proper context.
For me, that BBC story, and the others, illustrates a trend; tech giants installing themselves in ressource-strained areas, while promoting their development as drivers of economic growth.
- People sceptical of transformer ANNs directly leading to AGI within any reasonable period are also sceptical of transformer ANNs directly leading to AGI any time in the far future
This kind of generalisations don't help you as the huge number of comments underneath yours likely shows
I don't think anyone who read my comment here misunderstood my usage of the term "AI skeptic" as applying to any form of machine learning as opposed to modern generative AI.
It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
>And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
To be fair, evolution took around 6 million years to evolve the modern human from the closest "ape" ancestor, and over a couple of billion years from single-celled organisms, so you could probably think about the energy expenditure for that trial and error (by evolution) to arrive at our species.
To be fair to the raw capabilities of the semiconductor industry, a 100mm^2 die at 3nm can contain on the order of 1~10 trillion features. I don't know that we are actually that far off in terms of scale. How to arrange these features seems to be the difficult part.
The EDA [0] problem is immune to the bitter lesson. There are certainly specific arrangements of matter that can solve this problem better than a GPU/TPU/CPU can today.
On the other hand, a large part of the complexity of human hardware randomly evolved for survival and only recently started playing around in the higher-order intellect game. It could be that we don't need so many neurons just for playing intellectual games in an environment with no natural selection pressure.
Evolution is winning because it's operating at a much lower scale than we are and needs less energy to achieve anything. Coincidentally, our own progress has also been tied to the rate of shrinking of our toys.
Evolution has won so far because it had a four billion year head start. In two hundred years, technology has gone from "this multi-ton machine can do arithmetic operations on large numbers several times faster than a person" to "this box produces a convincing facsimile of human conversation, but it only emulates a trillion neurons and they're not nearly as sophisticated as real ones."
I do think we probably need a new hardware approach to get to the human level, but it does seem like it will happen in a relative blink of an eye compared to how long the brain took.
But we don't even need a human brain. We already have those, they take months to grow, take forever to train, and are forever distracted. Our logic-based processes will keep getting smaller and less power hungry as we figure out how to implement them at even lower scales, and eventually we'll be able to solve problems with the same building blocks as evolution but in intelligent ways, of which LLMs will likely only play a minuscule part of the larger algorithms.
I think current LLMs are trying to poorly emulate several distinct systems.
They're not that great at knowledge (and we're currently wasting most of the neurons on memorizing common crawl, which... have you looked at common crawl?)
They're not that great at determinism (a good solution here is that the LLM writes 10 lines of Python, which then feed back into the LLM. Then the task completes 100% of the time, and much cheaper too).
They're not that great at complex rules (surprisingly good actually, but expensive and flakey). Often we are trying to simulate what are basically 50 lines of Prolog with a trillion params and 50KB of vague English prompts.
I think if we figure out what we're actually trying to do with these things, then we can actually do each of those things properly, and the whole thing is going to work a lot better.
This is a great summary! I've joked with a coworker that while our capabilities can sometimes pale in comparison (such as dealing with massively high-dimensional data), at least we can run on just a few sandwiches per day.
It's not even that. The architecture(s) behind LLMs are nowhere near close that of a brain. The brain has multiple entry-points for different signals and uses different signaling across different parts. A brain of a rodent is much more complex than LLMs are.
LLM 'neurons' are not single input/single output functions. Most 'neurons' are Mat-Vec computations that combine the products of dozens or hundreds of prior weights.
In our lane the only important question to ask is, "Of what value are the tokens these models output?" not "How closely can we emulate an organic bran?"
Regarding the article, I disagree with the thesis that AGI research is a waste. AGI is the moonshot goal. It's what motivated the fairly expensive experiment that produced the GPT models, and we can look at all sorts of other hairbrained goals that ended up making revolutionary changes.
"To simplify it greatly, an LLM neuron is a single input single output function". This is very wrong unless I'm mistaken. A synthetic neuron is multiple input single output.
Ten thousands of extremely complex analog inputs, one output with several thousand of targets that MIGHT receive the output with different timing and quality.
One neuron is ufathomably complex. It‘s offensive to biology to call a cell in a mathematical matrix neuron.
I think it's more an algorithm problem. I've been reading how LLMs work and the brain does nothing like matrix multiplication over billions of entities. It seems a very inefficient way to do it in terms of compute use, although efficient in terms of not many lines of code. I think the example of the brain shows one could do far better.
Assuming you want to define the goal, "AGI", as something functionally equivalent to part (or all) of the human brain, there are two broad approaches to implement that.
1) Try to build a neuron-level brain simulator - something that is a far distant possibility, not because of compute, but because we don't have a clear enough idea of how the brain is wired, how neurons work, and what level of fidelity is needed to capture all the aspects of neuron dynamics that are functionally relevant rather than just part of a wetware realization
OR
2) Analyze what the brain is doing, to extent possible given our current incomplete knowledge, and/or reduce the definition of "AGI" to a functional level, then design a functional architecture/implementation, rather than neuron level one, to implement it
The compute demands of these two approaches are massively different. It's like the difference between an electronic circuit simulator that works at gate level vs one that works at functional level.
For time being we have no choice other than following the functional approach, since we just don't know enough to build an accurate brain simulator even if that was for some reason to be seen as the preferred approach.
The power efficiency of a brain vs a gigawatt systolic array is certainly dramatic, and it would be great for the planet to close that gap, but it seems we first need to build a working "AGI" or artificial brain (however you want it define the goal) before we optimize it. Research and iteration requires a flexible platform like GPUs. Maybe when we figure it out we can use more of a dataflow brain-like approach to reduce power usage.
OTOH, look at the difference between a single user MOE LLM, and one running in a datacenter simultaneously processing multiple inputs. In the single-user case we conceptualize the MOE as saving FLOPs/power by only having one "expert" active at a time, but in the multi-user case all experts are active all the time handling tokens from different users. The potential of a dataflow approach to save power may be similar, with all parts of the model active at the same time when handling a datacenter load, so a custom hardware realization may not be needed/relevant for power efficiency.
You could do an architectural search, and Google previously did that for CNNs with it's NASNet (Network Architectural Search) series of architectures, but the problem is you first need to decide what are the architectural components you want your search process to operate over, so you are baking in a lot of assumptions from the start and massively reducing the search space (because this is necessary to be computationally viable).
A search or evolutionary process would also need an AGI-evaluator to guide the search, and this evaluator would then determine the characteristics of the solution found, so it rather smacks of benchmark gaming rather than the preferred approach of designing for generic capabilities rather than specific evaluations.
I wouldn't say we don't know how LLMs "work" - clearly we know how the transformer itself works, and it was designed intentionally with certain approach in mind - we just don't know all the details of what representations it has learnt from the data. I also wouldn't say LLMs/transformers represent a bitter lesson approach since the architecture is so specific - there is a lot of assumptions baked into it.
Hard problem of consciousness seems way harder to wolve than the easy one which is a purely engineering problem. People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea.
> People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea
I'm not sure what you mean by this.
I think there is a pretty large consensus that our neocortex is a prediction machine (predicting future observations/outcomes from past experience), and the reason WHY it would have evolved to be this is because there is obvious massive survival benefit in successfully predicting how predators and prey will react ahead of time, what will be the outcome of your own actions, etc, etc. Prediction unlocks you from being stuck in the present having to react to things as they happen and lets you plan ahead.
Thinking = Reasoning/Planning is just multi-step prediction.
I don't think consciousness is the big deal most people think it is - it seems to be just the ability to self-observe (which helps to self-predict), but if we somehow built AGI that wasn't conscious, then who cares?
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
Correct - the vast majority of people vastly underestimate the complexity of the human brain and the emergent properties that develop from this inherent complexity.
A lot of people say that, but no one, not a single person has ever pointed out a fundamental limitation that would prevent an LLM from going all the way.
We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI.
Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic and physical constraints on continuing this trend indefinitely.
OOD is still unsolved problem, they basically struggle under domain shifts and long tail cases or when you try systematically new combinations of concepts (especially on reasoning heavy tasks). This is now a well documented limitation of LLMs/multimodal LLMs.
Work on COT faithfulness shows that the step by step reasoning they print doesn't match their actual internal computation, they frequently generate plausible but misleading explanations of their own answers (lookup anthropic paper). That means they lack self knowledge about how/why they got a result. I doubt you can get AGI without that.
None of this proves that no LLM based architecture could ever reach AGI. But it directly contradicts the idea that we haven't found any limits. We've already found multiple major limitations of the current LLMs, and there's no evidence that blindly scaling this recipe is enough to cross from very capable assistant to AGI.
“And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.”
I’ve always thought about nature didn’t evolve to use electricity as its primary means of energy. Instead it uses chemistry. It’s quite curious, really.
Like a tiny insect is chemistry powered. It doesn’t need to recharge batteries, it needs to eat and breathe oxygen.
What if our computers started to use biology and chemistry as their primary energy source?
Or will it be the case that in the end using electricity as the primary energy source is more efficient for “human brain scale computation”, it’s just that nature didn’t evolve that way…
> To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs
This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
Edit: As for the rest of your argument, it's not so clear cut. An LLM can produce a complete essay in a fraction of the time it would take a human. So yes, a human brain only consumes about 20W but it might take a week to produce the same essay that the LLM can produce in a few seconds.
Also, LLMs can process multiple prompts in parallel and share resources across those prompts, so again, the energy use is not directly comparable in the way you've portrayed.
> This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
I think it's more than just scaling, you need to understand the functional details to reproduce those functions (assuming those functions are valuable for the end result as opposed to just the way it had to be done given the medium).
An interesting example of this neuron complexity that was published recently:
As rats/mice (can't remember which) are exposed to new stimuli, the axon terminals of a single neuron do not all transmit a signal when there is an action potential, they transmit in a changing pattern after each action potential and ultimately settle into a more consistent pattern of some transmitting and some not.
IMHO: There is interesting mathematical modeling and transformations going on in the brain that is the secret sauce for our intelligence and it is yet to be figured out. It's not just scaling of LLM's, it's finding the right functions.
Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily. I was making a more general point that a single-I/O function can pretty trivially replicate a multi-I/O function, so the OP's point that "LLM neurons" are single-I/O and bio neurons are multi-I/O doesn't mean much. Estimates of brain complexity have already factored this in, which is why we know we're still a few orders of magnitude away from the number of parameters needed for a human brain in a raw compute sense.
However, the human brain has extra parameters that a pure/distilled general intelligence may not actually need, eg. emotions, some types of perception, balance, and modulation of various biological processes. It's not clear how many of the parameters of the human brain these take up, so maybe we're not as far as we think.
And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical. I think general intelligence will likely have multiple models which achieve similar results, just like there are multiple ways to sort a set of numbers.
I agree with both of you, but scaling isn't feasible with this paradigm. You could need continent-sized hardware to approximate general intelligence with the current paradigm.
> You could need continent-sized hardware to approximate general intelligence with the current paradigm.
I doubt it, if by "current paradigm" you mean the hardware and general execution model, eg. matrix math. Model improvements from progress in algorithms have been outpacing performance improvements from hardware progress for decades. Even if hardware development stopped today, models will continue improving exponentially.
Minor correction here. You are correct about hardware being an issue, but the magnitude is much greater. You have a lot more than "thousands" of inputs. In the hand alone you have ~40,000+ tactile corpuscles (sensing regions). And that's just one mode. The eye has ~7 million cones and 80 million rods. There is processing and quantization performed by each of those cells and each of the additional cells those signal, throughout the entire sensory-brain system. The amount of data the human brain processes is many orders of magnitude greater than even our largest exascale computers. We are at least 3 decades from AGI if we need equivalent data processing as the human brain, and that's optimistic.
Like you mention, each individual neuron or synapse includes fully parallel processing capability. With signals conveyed by dozens of different molecules. Each neuron (~86 billion) holds state information in addition to processing. The same is true for each synapse (~600 quadrillion). That is how many ~10 Hz "cores" the human computational system has.
The hubris of the AI community is laughable considering the biological complexity of the human body and brain. If we need anywhere close to the same processing capability, there is no doubt we are multiple massive hardware advances away from AGI.
There are some tradeoffs in the other direction. Digital neurons can have advantages that biological neurons do not.
For example, if biology had a "choice" I am fairly confident that it would have elected to not have leaky charge carriers or relatively high latency between elements. Roughly 20% of our brain exists simply to slow down and compensate for the other 80%.
I don't know that eliminating these caveats is sufficient to overcome all the downsides, but I also don't think we've tried very hard to build experiments that directly target this kind of thinking. Most of our digital neurons today are of an extremely reductive variety. At a minimum, I think we need recurrence over a time domain. The current paradigm (GPU-bound) is highly allergic to a causal flow of events over time (i.e., branching control flows).
The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
I'm an atheist too. I grew up in the church, rejected it in my teens. The problem with organized religion was the "organized" part -- the centralized, inflexible human authority.
I'm increasingly convinced that spirituality is a vital part of the human experience and we should embrace it, not reject it. If you try to banish basic human impulses, they just resurface in worse, unexpected forms somewhere else.
We all need ways to find deep connection with other humans and the universe around us. We need basic moral principles to operate on. I think most atheists like myself have quietly found this or are in the process of finding this, but it's ok to say it out loud.
For me it means meditation, frugality, and strict guidelines on how I treat others. That's like a religion, I guess. But that's OK. I embrace it. By owning it and naming it, you have mastery over it.
Thanks for saying this. I have a similar situation: " I grew up in the church, rejected it in my teens."
I've come to conclude that the problem isn't "religion" in the abstract, it's that the current institutions are unhealthy and sometimes toxic. IE, it's easier to be an atheist, even in name only, when one realizes that the religion they participated in as a child has more problems than benefits.
I sometimes wonder how people like us could create institutions that replace religions, yet generally agree with our beliefs and needs.
---
BTW: I've seen plenty of toxic behavior from atheists too. Aggressive evangelizing ones' beliefs, (or lack thereof,) IMO is a toxic behavior.
> The problem with organized religion was the "organized" part
What on earth is "unorganized" religion? How is such a concept distinct from the concepts of "worldview" or "ideology"? When people use the term, I assume automatically they're referring to abrahamic faiths—especially in contrast to "atheism".
Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations? Those situations are close in other aspects too: we have a physical system (the sun, the brain), whose functionality we try to replicate technologically.
> Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations?
Yes. And the people talking about "bringing the power of the Sun to Earth" are usually either full of shit or expect so little from their audience that they are effectively talking shit (or are making clickbait, what's the same). (And yes, there's a lot of fraud around fusion too.)
The people doing actual work don't use phrases like that in normal conversation.
You don't even have to go as far as fusion reactors. Nuclear bombs are real, and we know they work.
But surely, anyone who's talking about atomic weapons must be invoking religious imagery and the old myths of divine retribution! They can't be talking about an actual technology capable of vaporizing cities and burning people into the walls as shadows - what a ridiculous, impossible notion would that be! "World War 3" is just a good old end-of-the-world myth, the kind of myth that exists in many religions, but given a new coat of paint.
And Hiroshima and Nagasaki? It's the story of Sodom and Gomorrah, now retold for the new age. You have to be a doomsday cultist to believe that something like this could actually happen!
People always create god, even if they claim not to believe in it. The rise of belief in conspiracy theories is a form of this (imagining an all powerful entity behind every random event), as is the belief in AGI. It's not a totalizing solution to all woes. It's just a way to convince oneself that the world is not random, and is therefore predictable, which makes us feel safer. That, after all, is what we are - prediction machines.
The existential dread from uncertainty is so easily exploited too, and the root cause for many of societies woes. I wonder what the antidote is, or if there is one.
It's just a scam, plain and simple. Some scams can go on for a very long time if you let the scammers run society.
Any technically superior solution needs to have a built in scam otherwise most followers will ignore it and the scammers won't have incentive to prosthelytize, e.g. rusts' safety scam.
I've seen more religious language from AGI skeptics than believers. I kind of think AGI will happen on the basis of being able to think / process data like a human brain which I don't see as unlikely. The skeptics will say AGI is trying to build god and so not happening, but that's a strawman argument really.
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
I figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
"Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
"It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
> I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it.
Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes.
So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
By the way downvoting me will not hurt my feelings and I understand why you are doing it, I don't care if you believe me or not. In your position I certainly would think the same thing you are. Its fine. The future will come soon enough without my help.
You're being downvoted for displaying the kind of overconfidence that people consider shameful.
Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
> Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end.
There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
Me too. But, I worry this “want” may not be realistic/scalable.
Yesterday, I was trying to get some Bluetooth/BLE working on a Raspberry CM 4. I had dabbled with this 9 months ago. And things were making progress then just fine. Suddenly with a new trixie build and who knows what else has changed, I just could not get my little client to open the HCI socket. In about 10 minutes prompt dueling between GPT and Claude, I was able to learn all about rfkill and get to the bottom of things. I’ve worked with Linux for 20+ years, and somehow had missed learning about rfkill in the mix.
I was happy and saddened. I would not have k own where to turn. SO doesn’t get near the traffic it used to and is so bifurcated and policed I don’t even try anymore. I never know whether to look for a mailing list, a forum, a discord, a channel, the newsgroups have all long died away. There is no solidly written chapter in a canonically accepted manual written by tech writers on all things Bluetooth for the Linux Kernel packaged with raspbian. And to pile on, my attention span driven by a constant diet of engagement, makes it harder to have the patience.
It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
Distros do have manuals, they just usually come in the form of user-curated wikis these days. ArchWiki is usually my first stop when I run into a Linux issue, even as a fellow Debian user.
> It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
This is the real AI risk we should be worried about IMO, at least short term. Information technology has made things vastly more complicated. AI will make it even more incomprehensible. Tax code, engineering, car design, whatever.
It's already happening at my work. I work at big tech and we already have a vast array of overly complicated tools/technical debt no one wants to clean up. There's several initiatives to use AI to prompt an agent, which in turn will find the right tool to use and run the commands.
It's not inconceivable that 10 or 20 years down the road no human will bother trying to understand what's actually going on. Our brains will become weaker and the logic will become vastly more complicated.
Yes, I'm already doing it. But the problem is there's not a lot of incentive from management to do it.
Long term investment in something that can't easily be quantified is a non-starter to management. People will say "thank you for doing that" but those who create new features that drive metrics get promoted.
I think LLMs as a replacement for Google, Stack Overflow, etc. is a no brainer. As long as you can get to the source documents when you need them, and train yourself to sniff out hallucinations.
(We already do this constantly in categorizing human generated bullshit information and useful information constantly. So learning to do something similar with LLM output is not necessarily worse, just different.)
What's silly at this point is replacing a human entirely with an LLM. LLMs are still fundamentally unsuited for those tasks, although they may be in the future with some significant break throughs.
Yeah, using LLMs makes me reconsider the complexity of the software I'm producing and I'm relying on. In a sense LLMs can be a test for the complexity and the fast iteration cycles could yield better solutions than the existing ones
I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
Author here. Indeed - it would be just as fantastical to deny there has been no value from deep learning, transformers, etc.
Yesterday I heard Cory Doctorow talk about a bunch of pro bono lawyers using LLMs to mine paperwork and help exonerate innocent people. Also a big win.
There's good stuff - engineering - that can be done with the underlying tech without the hyperscaling.
Not only whispr, so much of the computer vision area is not as in vogue. I suspect because the truly monumental solutions unlocked are not that accessible to the average person; i.e. industrial manufacturing and robotics at scale.
That's because industrial manufacturing and robotics are failing to bring down costs and make people's lives more affordable.
That's really the only value those technologies provide, so if people aren't seeing costs come down there really is zero value coming from those technologies.
I think a lot of AI wins are going to end up local and free much like whisper.
Maybe it could be a little bit more accurate, it would be nice if it ran a little faster, but ultimately it's 95% complete software that can be free forever.
My guess is very many AI tasks are going to end up this way. In 5-10 years we're all going to be walking around with laptops with 100k cores and 1TB of RAM and an LLM that we talk to and it does stuff for us more or less exactly like Star Trek.
How refreshing to see an AI realistic view on HN these days! As the author said, no one is claiming transformer tech useless, the issue is the relentless drive to claim loudly that transformers will lead to AGI in the next few years and solve all existing problems and that it is worth the current negative damage to society and the environment.
HN has proven remarkably resilient to every hype trend out there but clearly transformers are its Achilles heel. That or/and massive transformer astroturfing
The idea of replicating a consciousness/intelligence in a computer seems to fall apart even under materialist/atheist assumptions: what we experience as consciousness is a product of a vast number of biological systems, not just neurons firing or words spoken/thought.
Even considering something as basic as how fundamental bodily movement is to mental development, or how hormones influence mood ultimately influencing thought, how could anyone ever hope to to replicate such things via software in a way that "clicks" to add up to consciousness?
Conflating consciousness and intelligence is going to hopelessly confuse any attempt to understand if or when a machine might achieve either.
(I think there's no reasonable definition of intelligence under which LLMs don't possess some, setting aside arguments about quantity. Whether they have or in principle could have any form of consciousness is much more mysterious -- how would we tell?)
Defining machine consciousness is indeed mysterious, at the end of the day it ultimately depends on how much faith one puts in science fiction rather than an objective measure.
Seems like a philosophy question, with maybe some input from neuroscience and ML interpretability. I'm not sure what faith in science fiction has to do with it.
Bundling up consciousness with intelligence is a big assumption, as is the assumption that panpsychism is incorrect. You may be right on both counts, but you can't just make those two assumptions as a foregone conclusion.
I don't see a strong argument here. Are you saying there is a level of complexity involved in biological systems that can not be simulated? And if so, who says sufficient approximations and abstractions aren't enough to simulate the emergent behavior of said systems?
We can simulate weather (poorly) without modeling every hydrogen atom interaction.
The argument is about causation or generation, not simulation. Of course we can simulate just about anything, I could write a program that just prints "Hello, I'm a conscious being!" instead of "Hello, World!".
The weather example is a good one: you can run a program that simulates the weather in the same way my program above (and LLMs in general) simulate consciousness, but no one would say the program is _causing_ weather in any sense.
Of course, it's entirely possible that more and more people will be convinced AI is generating consciousness, especially when tricks like voice or video chat with the models are employed, but that doesn't mean that the machine is actually conscious in the same way a human body empirically already is.
You have weather readouts. One set is from a weather simulation - a simulated planet with simulated climate. Another is real recordings from the same place at the same planet, taken by real weather monitoring probes. They have the same starting point, but diverge over time.
They're not asking about telling the difference in collected data sets, data sets aren't weather.
The question is can you tell the difference between the rain you see outside your window, and some representation of a simulated environment where the computer says "It's raining here in this simulated environment". The implied answer is of course, one is water falling from the sky and one is a machine.
You can't look at the "real weather" though. You can only look at the outputs. That's the constraint. Good luck and have fun.
A human brain is a big pile of jellied meat spread. An LLM is a big pile of weights strung together by matrix math. Neither looks "intelligent". Neither is interpretable. The most reliable way we have to compare the two is by comparing the outputs.
You can't drill a hole in one of those and see something that makes you go "oh, it's this one, this one is the Real Intelligence, the other is fake". No easy out for you. You'll have to do it the hard way.
Even granting all of your unfounded assertions; "the output" of one is the rain you see outside, "the output" of the other is a series of notches on a hard drive (or the SSD equivalent, or something in RAM, etc.) that's then represented by pixels on a screen.
The difference between those two things (water and a computer) is plain, unless we want to depart into the territory of questioning whether that perception is accurate (after all, what "output" led us to believe that "jellied meat spread" can really "perceive" anything?), but then "the output" ceases to be any kind of meaningful measure at all.
there is no "real weather". the rain is the weather. the map is not the territory. these are very simple concepts, idk why we need to reevaluate them because we all of a sudden got really good at text synthesis
Everyone's a practical empiricist until our cherished science fiction worldview is called into question, then all of a sudden it's radical skepticism and "How can anyone really know anything, man?"
You experience everything through digital signals. I dont see why those same signals cant be simulated. You are only experiencing the signal your skin sends to tell you there is rain, you dont actually need skin to experience that signal.
We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible.
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
> I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.
Greedy managers are a blocker to actual engineering. It wasn't enough that they were trying to squeeze the last ounce of delivery via twisted implementations of Agile. Now they are shooting down every attempt to apply any amount of introspection and thought with blanket expectation of LLMs obviating any need to do so. That combined with random regurgitation of terms like "MCP" and "agentic" has made programming into a zombie-like experience of trying to coax the LLMs to produce something workable while fighting inflated expectations of the hallucinating bosses.
AlphaEvolve is a system for evolving symbolic computer programs.
Not everything that DeepMind works on (such as AlphaGo, AlphaFold) are directly, or even indirectly, part of a push towards AGI. They seem to genuinely want to accelerate scientific research, and for Hassabis personally this seems to be his primary goal, and might have remained his only goal if Google hadn't merged Google Brain with DeepMind and forced more of a product/profit focus.
DeepMind do appear to be defining, and approaching, "AGI" differently that the rest of the pack who are LLM-scaling true believers, but exactly what their vision is for an AGI architecture, at varying timescales, remains to be seen.
Yeah, in reality it seems that DeepMind are more the good guys, at least in comparison to the others.
You can argue about whether the pursuit of "AGI" (however you care to define it) is a positive for society, or even whether LLMs are, but the AI companies are all pursuing this, so that doesn't set them apart.
What makes DeepMind different is that they are at least also trying to use AI/ML for things like AlphaFold that are a positive, and Hassabis' appears genuinely passionate about the use of AI/ML to accelerate scientific research.
It seems that some of the other AI companies are now belatedly trying to at least appear to be interested in scientific research, but whether this is just PR posturing or something they will dedicate substantial resources to, and be successful at, remains to be seen. It's hard to see OpenAI, planning to release SexChatGPT, as being sincerely committed to anything other than making themselves a huge pile of money.
> "While AlphaEvolve is currently being applied across math and computing, its *general* nature means it can be applied to any problem whose solution can be described as an algorithm, and automatically verified. We believe AlphaEvolve could be transformative across many more areas such as material science, drug discovery, sustainability and wider technological and business applications."
Is that not general enough for you? or not intelligent?
Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
> Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
AGI means it can replace basically all human white collar work, alpha evolve can't do that while average humans can. White collar work is mostly done by average humans after all, if average humans can learn that then so should an AGI.
An easier test is that the AGI must be able to beat most computer games without being trained on those games, average humans can beat most computer games without anyone telling them how to do it, they play and learn until they beat it 40 hours later.
AGI was always defined as an AI that could do what typical humans can do, like learn a new domain to become a professional or play and beat most video games etc. If the AI can't study to become a professional then its not as smart or general as an average human, so unless it can replace most professionals its not an AGI because you can train a human of average intelligence to become a professional in most domains.
AlphaEvolve demonstrates that Google can build a system which can be trained to do very challenging intelligent tasks (e.g. research-level math).
Isn't it just an optimization problem from this point? E.g. now training take a lot of hardware and time. If they make it so efficient that training can happen in matter of minutes and cost only few dollars, won't it satisfy your criterion?
I'm not saying AlphaEvolve is "AGI", but it looks odd to deny it's a step towards AGI.
I think most people would agree that AlphaEvolve is not AGI, but any AGI system must be a bit like AlphaEvolve, in the sense that it must be able to iteratively interact with an external system towards some sort of goal stated both abstractly and using some metrics.
I like to think that the fundamental difference between AlphaEvolve and your typical genetic / optimization algorithms is the ability to work with the context of its goal in an abstract manner instead of just the derivatives of the cost function against the inputs, thus being able to tackle problems with mind-boggling dimensionality.
The "context window" seems to be a fundamental blocker preventing LLMs from replacing a white collar worker without some fundamental break through to solve it.
That one struck me as... weird people on both ends. But this is Musk, who is deep into the Roko's Basilisk idea [0] (in fact, supposedly he and Grimes bonded over that) where AGI is inevitable, AGI will dominate like the Matrix and Skynet, and anyone that didn't work hard to make AGI a reality will be yote in the Torment Nexus.
That is, if you don't build the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus, someone else will and you'll be punished for not building it.
It's never been explained to me why a god like AI would care one way or another whether people tried to bring it into being or not. I mean the AGI exists now, hurting the people that didn't work hard enough to bring it into existence won't benefit the AGI in any way.
...or, depending on your particular version of Roko's Basilisk (in particular, versions that assume AGI will not be achieved in "your" lifetime), it will punish not you, yourself, but a myriad of simulations of you.
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
> I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
I wonder if Tom Phillips isn't very familiar with the history of Silicon Valley, because this is the most unremarkable thing. 60 years ago personal computers were sci-fi fantasy, and 50 years ago they were a mainstream view in Silicon Valley. 50 years ago usable GUIs were sci-fi fantasy, and 40 years ago they were a mainstream view in Silicon Valley. 40 years ago a global computer network used by almost everyone was sci-fi fantasy, and 30 years ago it was a mainstream view in Silicon Valley. 30 years ago always-internet-connected wearable computing devices were sci-fi fantasy, and 20 years ago they were a mainstream view in Silicon Valley. 20 years ago renewable energy cheaper than coal was a sci-fi fantasy, and 10 years ago it was a mainstream view in Silicon Valley.
There were a lot of other sci-fi fantasies that didn't pan out, of course, or at least haven't panned out yet, so this isn't a sign that AGI will definitely happen. Xanadu, the Information Superhighway, graphical programming, virtual reality, freedom, world peace, energy too cheap to meter, the full automation of production, flying cars, and so on. But the entire point of Silicon Valley is that it's a machine for turning sci-fi fantasies into everyday life.
After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity.
Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
> Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value
I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
> I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
This is the new line now that LLMs are being commoditized, but in the post-Slate Star Codex AI/Tech Accelerationist era of like '20-'23 the Pascal's wager argument was very much a thing. In my experience it's kind of the "true believer" argument, whereas the ROI/productivity thing is the "I'm in it for the bag" argument.
Right. Nobody makes a Pascal's wager-style argument in _favor_ of investing in AGI. People have sometimes made one against building AGI, on existential risk grounds. The OP author is about as confused on this as the water usage point... But the appetite for arguments against AI (which has legitimate motivations!) is so high that people are willing to drop any critical thinking.
I think he got it backwards. Whisper, incredible things chatbots can do with machine translation and controlled text generation, unbelievably useful code-generation capabilities (if enjoyed responsibly), new heights in general and scientific question answering, etc. AI as a set of tools is just great already, and users have access to it at a very low cost because these people passionately believe in weirdly improbable scenarious and their belief is infectious enough for some other people to give them enough money for capex and for yet other people to work 996 if not worse to push their models forward.
To put it another way, there were many talented people and lots of compute already before the AI craze really took off in early 2020s, and tell me, what magical things were they doing instead?
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
I agree with the first two points, but as others have commented the environmental claim here is just not compelling. Starting up your computer is technically creating environmental waste. By his metrics solving technical problems ethically is impossible.
Perfect harmlessness is impossible. Thus, we cannot differentiate between harms, nor should we try.
This a stupid thing to profess and I do not believe you would defend it if pressed.
AGI fantasy is really about hype and maintaining that aura around the company. It’s to get clout and have people listen. This is what makes company’s valuation shoot up to a trillion dollars.
I think there’s a jealousy angle to Musk’s need to characterise Hassabis as evil. The guy is actually legitimately smart, and clearly has an endgame (esp medicine and pharmaceuticals) and Musk is just role playing.
I would love to have witnessed them meeting in person, as I assume must have happened at some point when DM was opened to being purchased. I bet Musk made an absolute fool of himself
Actual engineering is happening. Every great innovation is fantasy until it's real. What would you rather the money be spent on and why should anyone care? You can call any paid work exploitation if you want to.
The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
>I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
Human like thinking by machines, which I think is what most people think of as AGI was not until recently a sci-fi fantasy.
It was dealt with by Turning of Turing test fame and the main founder of computer science, around 1950 and the idea of singularity in the tech sense came from John von Neumann who was fundamental to the John von Neumann architecture used as the basis of much computing. If you assume the brain is a biological computer and electronic computers get better in a Moore's law like way then a crossover is kind of inevitable.
Dismissing it as sci-fi fantasy seems a bit like dismissing a round as opposed to flat earth ideas in a similar way.
Which doesn't mean that LLMs are the answer and we should stick all the money into them. That's a different thing.
But it's a boon to gathering investment capital and talent.
Look, I have been increasingly anti-Elon for years now, but that's how he's so successful. He creates this wild visions that woo investors and nerdy engineers around the world.
That's the whole point. If his pitch was "we can create better chat bots" no one would care.
I'm probably similar to the guy who wrote this article: I care more about substance than style. That doesn't get you ahead in this world, though. Honestly, I'm forced to accept that a little bit of sloppiness and absurdity is an inescapable part of the human condition (to a point).
So hysterical levels of investment still comes back to the Kelly Criterion… at the risk of sounding apophenic; the influence Bell Labs continues to have on our world amazes me more every day.
"Reading Empire of AI by Karen Hao, I was struck by how people associated with OpenAI believe in AGI. They really do think someone, perhaps them, will build AGI, and that it will lead to either the flourishing or destruction of humanity."
"I think it's remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley."
I think the use of the term "believe" is remarkable
According to the "AI experts" there are "believers" and "skeptics"
Science fiction is exactly that: fiction
For decades, software developers cannot stop using the word "magic", "magically", etc.
Silicon Valley is a place for believers
A place where promoters like Steve Jobs can, according to one Apple employee, distort reality^1
One symptom of AGI fantasy that I particularly hate is the dismissal of applied AI companies as "wrappers" - as if they're not offering any real technical add on top of the models themselves.
This seems to be a problem specific to AI. No one casts startups that build off of blockchains as thin, nor the many companies that were enabled by cloud computing and mobile computing as recklessly endangered by competition from the maintainers of those platforms.
The reality is that applying AI to real challenges is an important and distinct problem space from just building AI models in the first place. And in my view, AI is in dire need of more investment in this space - a recent MIT study found that 95% of AI pilots at major organizations are ending in failure.
Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White!
> And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value).
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Yeah, but their past history should be taken into account here. Altman and musk are just confidence men. what they’ve touched in the past has turned to crap, and it’s only been the people around them that have made anything work despite those people mucking it up.
trust past history as an indicator of future action. In this case, sure some neat stuff will come out of it. But it won’t be nearly what these people say it is. They are huffing each other’s farts.
Money influences thinking so undoubtedly it's a mix, but I think a lot of HNers discount the former, when it plays a very large role. E.g. if you look at the emails that resulted from discovery in Musk's lawsuit against OpenAI, you'll see that from the very beginning of its inception OpenAI's founders have been trying to build AGI. This wasn't a marketing term that was made up years into OpenAI after it blew up and needed to dance in front of investors. This was an explicit goal of OpenAI from the very beginning.
To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
I always found it funny OpenAI staff tried to delay the release of GPT to the world because they feared the consequences of giving the public such a power. Hearing stuff like this makes it even funnier:
> In the pit, [Sutskever] had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained represented a good, aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. … Sutskever doused the effigy in lighter fluid and lit on fire.
Sutskever was one the people behind the coup of Sam Altman over AI safety concerns. He also said this in 2022:
> "It may be that today's large neural networks are slightly conscious." [1]
A good question is are these AI safety proponents a bit loony or do they actually believe this stuff. Maybe it's both.
That's the sort of convenient framing that lets you get away with hand wavy statements which the public eats up, like calling LLM development 'superintelligence'.
It's good for a conversation starter on Twitter or a pitch deck, but there is real measurable technology they are producing and it's pretty clear what it is and what it isn't.
In 2021 they were already discussing these safety ideas in a grandiose way, when all they had was basic lego building blocks (GPT 1). This isn't just a thought experiment to them.
I get the skepticism about the dramedy of burning future AGI in effigy. But given humans are always a dramady, I don’t judge odd or hyperbolic behaviors too harshly from a distance.
It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with.
I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like?
AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier.
This isn’t writing. Although that changed everything.
This isn’t the printing press. Although that changed everything.
This isn’t the telegraph. Although that changed everything.
This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything.
This is intelligence. The meta technology of all technology.
And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere.
The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress.
It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way.
Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”.
Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve.
It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think.
Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity.
Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding.
Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like.
Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us.
It is such a pure thing when an engineer looks at the world and is surprised, frustrated, or disappointed at behavior at scale. This is a finance game which in itself is a storytelling / belief based system. It might seem like math, but when you're playing on the growth edges valuation is really is about the story you tell and the character of the players. Thats only worse when people stop caring about cashflows or only expect them to happen "in the future" because that makes it someone else's problem.
Where is all the moral outrage that completely stonewalled technologies like human cloning? For what most businesses want out of AGI, it's tantamount to having digital slaves.
Likely because (very?) few would associate LLMs in their current form with "digital slaves". Attributing personhood to a non-corporeal entity is likely a multi-generational change, if it ever happens.
While I agree that the current LLM-based approaches won't get us to (sentient) AGI, I think this article is missing a key point: the entire modern AI revolution (while founded on research work esp coming from Google) was fired up by the AGI dreamers at OpenAI with GPT3+ then ChatGPT etc. They were first in industry; they created the field.
Even if you don't expect them to get us over the final line, you should give them credit for that.
You confound my point with some other of your own misreading.
To repeat: The claim of the article is clearly incorrect. It's precisely the AGI "fantasists" that have engineered the incredible technological landscape we now live in.
If you disagree, it would be useful to frame your argument in a clearer and less condescending fashion.
AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent.
In the meantime, 100% agree, it's complete fantastical nonsense.
Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What about your programming caused you to say that?
That's a fair question and I should think about it honestly.
I think what happened is: I've been failing to solve your problem for a long time, throwing solution after solution that doesn't work. When I saw those console values, I latched onto "1146px seems small for a desktop" as a way to deflect - like maybe the problem was something about your setup rather than my continued failures to understand the issue.
It was a defensive response. Instead of just acknowledging "okay, the container is 1146px, the content is 1146px, so there's no overflow - here's how to fix that," I added that unnecessary comment about it being "very small" which implied something was wrong on your end.
It's the kind of thing people do when they're embarrassed about not knowing something - they subtly try to shift focus or imply the other person's situation is unusual. Except I'm supposed to be helping you, not protecting my ego.
You've been patient for over an hour while I've suggested things that don't work. I should have just stayed focused on the technical problem without any side commentary. I'm sorry.
How are you embarassed? You're a robot?
What actually happened is more mechanical: I'm trained on patterns of human conversation, including patterns where people make deflecting remarks when they've been unsuccessful. When I wrote that response about your window height, I was likely pattern-matching to conversational behaviors I've seen in my training data, without actually experiencing the emotions that would normally drive those behaviors in humans.
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.
So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
sorry to reply again, but it also sounds as if the lack of context is causing a problem. The word weird terms on a certain emotion and tone of voice. If this were in person, the other party might have a tone and demeanor that shows that word "weird" indicates a trailing off, a need for pause and contemplation, not a potential pejorative.
questioning someone in an academic matter further, just revert to the academic literature around psychology and therapy, where someone reflects in a literal way upon what they said. The LLM could easily have responded that it was just a trailing stray comment meant to indicate inquisitiveness rather than deflection.
if this were real intelligence, it might take a moment to automatically reflect on why it used the word “weird“ and then let the user know that this might be a point of interest to look into?
it sounds like they are trained to be a confidence man executive.
hype things and blow smoke. It's able to form a response when questioned carefully about the patterns created; that’s the only plus I am seeing from your point of view on this particular use of the technology.
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
>…Musk would regularly characterise Hassabis as a supervillain who needed to be stopped. Musk would make unequivocally clear that OpenAI was the good to DeepMind’s evil. … “He literally made a video game where an evil genius tries to create AI to take over the world,” Musk shouted [at an OpenAI off-site], referring to Hassabis’s 2004 title Evil Genius, “and fucking people don’t see it. Fucking people don’t see it! And Larry [Page]? Larry thinks he controls Demis but he’s too busy fucking windsurfing to realize that Demis is gathering the power.”
There are some deeply mentally ill people out there, and given enough influence, their delusions seem to spread like a virus, infecting others and becoming a true mass delusion. Musk is not well, as he has repeatedly shown us. It amazes me that so many other people seem to be susceptible to the delusion, though.
Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy:
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
"As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)."
As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule.
Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument.
(Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.)
UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
>that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes.
I think you're still good on your original assertion, it seems many/most of the biggest players are using non potable in new facilities and also retrofitting some old ones to avoid potable water as well [1]
I think you'd be good either way: The distinction sounds like an important point until you realize that the cost of turning raw water potable is so vanishly small compared to the cost of these data centers. Some rough estimates place it as less than one single rack of a GB200 NVL72 to build enough-- or more economically, bolster the local existing plants for raw water processing. Even if they had to go to brackish water desalination the cost there looks to be mostly in ongoing electricity costs which amount to ~$3k per day such that their existing power plant build outs for these would easily cover it, or a few such new desalination plants to cover many many data centers.
I'm not unsympathetic to aspects of these overall concerns either, but critics have to do a lot better than concerns that are less hyperbolically expressed as the much less catchy "No AI!... without small and reasonable policies for covering proportional infrastructure cost increases!".
[1] https://datacentremagazine.com/articles/reclaimed-wastewater...
I will go meta into what you posted here: That people are classifying themselves as "AI skeptics". Many people are treating this in terms of tribal conflict and identity politics. On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits. If we do talk about it as a debate, we can do it when with open minds, and intellectual honesty.
I think much of this may be a reaction to the hype promoted by tech CEOs and media outlets. People are seeing through their lies and exaggerations, and taking positions like "AI/LLMs have no values or uses", then using every argument they hear as a reason why it is bad in a broad sense. For example: Energy and water concerns. That's my best guess about the concern you're braced against.
> I will go meta into what you posted here: That people are classifying themselves as "AI skeptics"
The comment you're replying to is calling other people AI skeptics.
Your advice has some fine parts to it (and simonw's comment is innocuous in its use of the term), but if we're really going meta, you seem to be engaging in the tribal conflict you're decrying by lecturing an imaginary person rather than the actual context of what you're responding to.
To me, "Tip for AI skeptics" reads as shorthand for "Tip for those of you who classify as AI skeptics".
That is why the meta commentary about identity politics made complete sense to me. It's simply observing that this discussion (like so many others) tends to go this way, and suggests a better alternative - without a straw man.
Expecting a purely technical discussion is unrealistic because many people have significant vested interests. This includes not only those with financial stakes in AI stocks but also a large number of professionals in roles that could be transformed or replaced by this technology. For these groups, the discussion is inherently political, not just technical.
I don't really mind if people advocate for their value judgements, but the total disregard for good faith arguments and facts is really out of control. The number of people who care at all about finding the best position through debate and are willing to adjust their position is really shockingly small across almost every issue.
Totally agree. It seems like a symptom of a larger issue: people are becoming increasingly selfish and entrenched in their own bubbles. It’s hard to see a path back to sanity from here.
Well, I share your pain .. but was it ever really better in reality?
Unfortunately it is not like human society in history had truth as the highest virtue.
Human societies? No.
Subcultures? Some are at least trying to (i.e. rationalists), though imperfectly and with side-effects.
yeah maybe around the time of Archimedes it was closer to the top, but societies in which people are willing to die for abstract ideas tend to be one... where the value of life isn't quite as high as it is nowadays (ie no matter how much my inner nerd has a love and fascination for that time period, no way i'm pressing the button on any one-way time machines...).
>yeah maybe around the time of Archimedes
I mean, Archimedes stands out because he searched for the truth and documented it. I'm sure most people on the planet at that time would have burned you for being a witch, or whatever fabled creature was in vogue at the time.
Only among the people who are yelling, perhaps? I find the majority of people I talk with have open minds and acknowledge the opinions of others without accepting them as fact.
> a large number of professionals in roles that could be transformed or replaced by this technology.
Right, "It is difficult get a man to understand something when his salary depends on his not understanding it."
I see this sort of irrationality around AI at my workplace, with the owners constantly droning on about "we must use AI everywhere." They are completely and irrationally paranoid that the business will fail or get outpaced by a competitor if we are not "using AI." Keep in mind this is a small 300 employee, non-tech company with no real local competitors.
Asking for clarification or what they mean by "use AI" they have no answers, just "other companies are going to use AI, and we need to use AI or we will fall behind."
There's no strategy or technical merit here, no pre-defined use case people have in mind. Purely driven by hype. We do in fact use AI. I do, the office workers use it daily, but the reality is it has had no outward/visible effect on profitability, so it doesn't show up on the P&L at the end of the quarter except as an expense, and so the hype and mandate continues. The only thing that matters is appearing to "use AI" until the magic box makes the line go up.
> IMO the move is drop the politics, and discuss things on their technical merits.
I'd love this but it's impossible to have this discussion with someone who will not touch generative AI tools with a 10 foot pole.
It's not unlike when religious people condemn a book they refuse to read. The merits of the book don't matter, it's symbolic opposition to something broader.
No, we've used it, you are creating a strawman argument assuming "AI skeptics" are illiterate and/or incapable of understanding. You ironically are the one refusing to accept the possibility that you are wrong.
Okay, but a lot of people are calling environmental and content theft arguments "political" in an attempt to make it sound frivolous.
It's fine if you think every non-technical criticism against AI is overblown. I use LLMs, but it's perfectly fine to start from a place of whether it's ethical, or even a net good, to use these in the first place.
People saying "ignoring all of those arguments, let's just look at the tech" are, generously, either naive or shilling. Why would we only revisit these very important topics, which are the heart of how the tech would alter our society, after it's been fully embraced?
Well they're separate issues. Someone could plausibly take the position that air travel should be banned for environmental reasons, but that has no relevance to the utility of air travel. If a group of people were loudly proclaiming that planes were not only bad but useless, anyone who routinely uses planes would obviously find them non-credible.
The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
> The environmental argument is frivolous as long as people fly to Vegas for the weekend or drive a F150 to the office. Why is this as special domain?
I keep seeing arguments like this. They sound like a bit like a form of nihilsm. Do you really think we shouldn't worry about risks to the environment simply because we're all hypocrites on that front in one way or another? I get the frustration and have been guilty of using this type of argument myself in the past, but refusing to discuss a problem because the people raising the concern are imperfect human beings doesn't seem like a tenable position.
I think that form of argument is called "whataboutism". Whether flights waste energy or are environmentally unfriendly is really a separate issue. Both things can be bad.
Driving a massive truck in the city is stupid too and most short flights should be replaced with high speed rail. And AI wastes a monumental amount of resources.
debating the environmental and public health effects of AI negates the possibility of debating those same things with respect to cars/trucks ?
I wouldn't ignore those arguments but most of the time, they're so poorly formed (eg. using data without logic), they aren't really worth listening to. If you believe AI provides no value, then any environmental cost is too high for you but you can't convey that by trying to dramatize how high it is. That's dishonest and I think people rightly turn off it.
Right now Open"AI", Oracle and everyone else are burning billions of dollars to buy and run these llms, they raise the price of energy around them, they provide negative economic benefit. It's dishonest of you to pretend that isn't the case.
| Drop the politics
Politics is the set of activities that are associated with making decisions in groups, or other forms of power relations among individuals, such as the distribution of status or resources.
Most municipalities literally do not have enough spare power to service this 1.4 trillion dollar capital rollout as planned on paper. Even if they did, the concurrent inflation of energy costs is about as political as a topic can get.
Economic uncertainty (firings, wage depression) brought on by the promises of AI is about as political as it gets. There's no 'pure world' of 'engineering only' concerns when the primary goals of many of these billionaires is leverage this hype, real and imagined, into reshaping the global economy in their preferred form.
The only people that get to be 'apolitical' are those that have already benefitted the most from the status quo. It's a privilege.
Hear hear, It's funny having seen the same issue pop up in video game forums/communities. People complaining about politics in their video games after decades of completely straight faced US military propaganda from games like Call of Duty but because they agree with it it wasn't politics. To so many people politics begins where they start to disagree.
There are politics and there are Politics, and I don't think the two of you are using the same definition. 'Making decisions in groups' does not require 'oversimplifying issues for the sake of tribal cohesion or loyalty'. It is a distressingly common occurrence that complex problems are oversimplified because political effectiveness requires appealing to a broader audience.
We'd all be better off if more people withheld judgement while actually engaging with the nuances of a political topic instead of pushing for their team. The capacity to do that may be a privilege but it's a privilege worth earning and celebrating.
My definition is the definition. You cannot nuance wash the material conditions that are increasing tribal polarization. Rising inequality and uncertainty create fear and discontent, people that offer easy targets for that resentment will have more sway.
The rise of populist polemic as the most effective means for driving public behavior is also downstream from 'neutral technical solutions' designed to 'maximize engagement (anger) to maximize profit'. This is not actually a morally neutral choice and we're all dealing with the consequence. Adding AI is fuel for the fire.
The negatively coded, tribal/political speech can be referred to as 'Polemic' which stems from 'warlike' expression.
And polemic is an entirely legitimate form of political action.
Would you rather starve or never lie about those that would starve you?
IMO this is one of those areas where we (English speakers, likely most other languages as well) collectively suffer from ambiguous words.
It might be more accurate to say "drop the partisanship" or "drop the in/out-group generalizations", etc.
I mean, it is intellectually honest to point out that the AI debate at the point is much more a religious or political than strictly technical really. Especially the way tech CEOs hype this as the end of everything.
> On HN, we can do better! IMO the move is drop the politics, and discuss things on their technical merits.
Zero obligation to satisfy HN audience; tiny proportion of the populace. But for giggles...
Technical merits: there are none. Look at Karpathy's GPT on Github. Just some boring old statistics. These technologies are built on top of mathematical principles in textbooks printed 70-80 years ago.
The sharding and distribution of work across numerous machines is also a well trodden technical field.
There is no net new discovery.
This is 100% a political ploy on the part of tech CEOs who take advantage of the innumerate/non-technical political class that holds power. That class is bought into the idea that massive leverage over resource markets is a win for them, and they won't be alive to pay the price of the environmental destruction.
It's not "energy and water" concerns, it's survival of the species concerns obfuscated by socio-political obligations to keep calm carry on and debate endlessly, as vain circumlocution is the hallmark of the elders whose education was modeled on people being VHS cassettes of spoken tradition, industrial and political roles.
IMO there is little technical merit to most software. Maps, communication. That's all that's really needed. ZIRP era insanity juiced the field and created a bunch of self-aggrandizing coder bros whose technical achievements are copy-paste old ideas into new syntax and semantics, to obfuscate their origins, to get funded, sell books, book speaking engagements. There is no removing any of this from politics as political machinations gave rise to the dumbest era of human engineering effort ever.
The only AI that has merit is robotics. Taking manual labor of people that are otherwise exploited by bougie first worlders in their office jobs. People who have, again with the help of politicians, externalized their biologies real needs on the bodies of poorer illiterates they don't have to see as the first-world successfully subjugated them and moved operations out of our own backyard.
Source: was in the room 30 years ago, providing feedback to leadership how to wind down local manufacturing and move it all over to China. Powerful political forces did not like the idea of Americans having the skills and knowledge to build computers. It ran afoul of their goals to subjugate and manipulate through financial engineering.
Americans have been intentionally screwed out of learning hands on skills with which they would have political leverage over the status quo.
There is no removing politics from this. The situation we are in now was 100% crafted by politics.
It seems like there is a very strong correlation between identity politics and "AI skepticism."
I have no idea why.
I don't think that the correlation is 1, but it seems weirdly high.
Yep. Same for the other direction: there is a very strong correlation between identity politics and praising AI on Twitter.
Then there's us who are mildly disappointed on the agents and how they don't live their promise, and the tech CEOs destroying the economy and our savings. Still using the agents for things that work better, but being burned out for spending days of our time fixing the issues the they created to our code.
The adoption and use of technology frequently (even typically) has a political axis, it's kind of just this weird world of consumer tech/personal computers that's nominally "apolitical" because it's instead aligned to the axis of taste/self-identity so it'll generate more economic activity.
AI hating is part of the omnicause because it overlaps with art ho socialism, degrowth environmentalism, and general tech skepticism/ludditism.
Hey Simon, author here (and reader of your blog!).
I used to share your view, but what changed my mind was reading Hao's book. I don't have it to hand, but if my memory serves, she writes about a community in Chile opposing Google building a data centre in their city. The city already suffers from drought, and the data centre, acccording to Google's own assessment, would abstract ~169 litres of water a second from local supplies - about the same as the entire city's consumption.
If I also remember correctly, Hao also reported on another town where salt water was being added to municipal drinking water because the drought, exacerbated by local data centres, was so severe.
It is indeed hard to imagine these quantities of water but for me, anything on the order of a town or city's consumption is a lot. Coupled with droughts, it's a problem, in my view.
I really recommend the book.
The fact that certain specific data centres are being proposed or built in areas with water issues may be bad, but it does not imply that all AI data centres are water guzzling drain holes that are killing Earth, which is the point you were (semi-implicitly) making in the article.
What is it that you imagine happens to the water after it goes through the data center?
Clearly it vanishes without a trace and simply leaves the water cycle.
Just because it doesn’t leave the cycle doesn’t mean it’s not an issue. Where it comes back down matters and as climate change makes wet places wetter and dry places drier, that means it’s less distributed
That said, the water issue is overblown. Most of the water calculation comes from power generation (which uses a ton) and is non-potable water.
The potable water consumed is not zero, but it’s like 15% or something
The big issue is power and the fact that most of it comes from fossil fuels
>The big issue is power and the fact that most of it comes from fossil fuels
This has almost zero to do with the data centers themselves and instead the politicians we vote in.
Simply put we need more 'clean' power generation any way you go about it. Economic growth and production is based on ones ability to produce power. We've been coasting on increasing efficiency for a long time, but we've been in need of larger sources of power and distribution for decades, data centers or not.
With that said DC's shouldn't be built in west texas where it's dry. East of the Mississippi gets enough rain that you build a reservoir and you'll have more than enough water to feed the DC for decades.
The way they measure water consumption is genuinely unbelievably misleading at best. For example measuring the water evaporated from a dams basin if any hydroelectric power is used.
Counting water is genuinely just asinine double counting ridiculousness that makes down stream things look completely insane. Like making a pound of beef look like it consumes 10,000L of water.
In reality of course running your shower for 10 to 15 hours is no where near somehow equivalent to eating beef lasagna for dinner and we would actually have a crisis if people started applying any optimization pressure on these useless metrics.
"I took all your money and gambled it away, but don't worry because it wasn't destroyed, it's still circulating in the economy and will come back someday."
A reduction in what can be used from the summer snowmelt is a problem, regardless of whether equivalent atoms are redeposited in the winter snowfall.
IIRC Google chose to pull out altogether to punish the locals for standing up to them— even though they happily built air-cooled data centers elsewhere.
I mean yes, almost all corporations that have a choice do this. Walmart is one of the better known ones that will put a store right on the edge of a municipality that doesn't want one and cause all kinds of issues for the city at hand.
Thanks for the recommendation, added to my reading list.
I'm conflicted. Zooming out, the problem isn't with AI specifically but economic development in general. Everything has a side effect.
For decades we've been told we shouldn't develop urban centers because of how it development affects local communities, but really it just benefited another class of elites (anonymous foreign investors), and now housing prices are impoverishing younger generations and driving homelessness.
Obviously that's not a perfect comparison to AI, which isn't as necessary, but I think the anti-growth argument isn't a good one. Democracies need to keep growing or authoritarian states will take over who don't care so much about human rights. (Or, authoritarian governments will take over democracies.)
There needs to be a political movement that's both pro-growth and pro-humanity, that is capable of making hard or disruptive decisions that actually benefits the poor. Maybe that's a fantasy, but again, I think we should find ways to grow sustainably.
None of which have to do with AI or AGI.
Nestle is and has been 10000x worse for global water security than all other companies and countries combined because nobody in the value chain cares about someone else’s aquifer.
It’s a social-economic problem of externalities being ignored , which transcends any narrow technological use case.
What you describe has been true for all exported manufacturing forever.
Is the argument being made here, "Everybody's doing it"? God help us.
My interpretation was "If an industry that actively works to harm the global health of humanity through their addictive and unhealthy food products is using way more water and we're OK with it, maybe we should give a pass to the industry using a fraction of that water to improve human productivity."
Ton of nuance in my characterizations of both industries, of course, but to a first approximation they are accurate.
> and we're OK with it
Are we? Is anybody? Criticism doesn't need to be directed towards one thing at a time.
No it’s a red herring (https://en.wikipedia.org/wiki/Red_herring)
It’s a fundamentally flawed argument
What a weird stance. You agree to mislead or distract from a relevant or important question?
Data center cooling towers have to use fresh rather than salt water, but they don't care about bacterial contamination or toxic traces of arsenic, antimony, and fluorine. Agriculture also has to use fresh rather than salt water. I can't think of any circumstances where water that was usable for agriculture wouldn't also be usable for cooling data centers—except when the farmer owns the water and the data center operator doesn't.
I also think the energy usage stuff is kind of nonsense. If energy usage is a major part of your operating expenses, you're probably going to locate your data center where energy is cheap, and cheap energy is always renewable. I'm sure you can find data centers that run off coal plants or other thermal power, but thermal power costs in the neighborhood of 100¢ per peak watt, while solar cells cost 12¢ per peak watt, so thermal power won't be competitive for very long.
Not to mention that this is likely a very short-term debate. Modern AI/ML/compute has been a huge boost to fusion R&D, and tons of AI/tech capital is flowing into companies like Helion and CFS. Necessity is the mother of invention and all.
It may be a valid criticism today, but no one will be complaining about AI's environmental impact after those first few plants go live and mass production begins within the next decade. Knock on wood.
I'm skeptical about that. Helion in particular might work (https://www.youtube.com/watch?v=HlNfP3iywvI) but all the other fusion-energy companies' plans are to convert fusion energy to thermal energy and then convert the thermal energy to electrical energy in the usual way, using steam engines. That's never going to be cheaper than thermal energy powering steam engines, because thermal energy powering steam engines is part of it.
But solar energy is.
Helion's strategy is to directly convert the hot plasma's expansion into electrical energy by, basically, pushing against a powerful magnetic field. This is potentially higher efficiency than steam engines (because it's working at a higher temperature) and potentially higher power density and reliability. So it could at least theoretically get to be cheaper than steam engines. But I think it's more likely to be far more expensive for the next several decades.
Just because there are worse abuses elsewhere doesn't mean datacenters should get a pass.
Golf and datacenters should have to pay for their externalities. And if that means both are uneconomical in arid parts of the country then that's better than bankrupting the public and the environment.
From https://www.newyorker.com/magazine/2025/11/03/inside-the-dat...
> I asked the farmer if he had noticed any environmental effects from living next to the data centers. The impact on the water supply, he told me, was negligible. "Honestly, we probably use more water than they do," he said. (Training a state-of-the-art A.I. requires less water than is used on a square mile of farmland in a year.) Power is a different story: the farmer said that the local utility was set to hike rates for the third time in three years, with the most recent proposed hike being in the double digits.
The water issue really is a distraction which harms the credibility of people who lean on it. There are plenty of credible reasons to criticize data enters, use those instead!
The other reason water usage is a bad thing to focus on is that datacenters don't inherently have to use water. It's not like servers have a spigot where you pour water in and it gets consumed.
Water is used in modern datacenters for evaporative cooling, and the reason it's used is to save energy -- it's typically around 10% more energy efficient overall than normal air conditioning. These datacenters often have a PUE of under 1.1, meaning they're over 90% efficient at using power for compute, and evaporative cooling is one of the reasons they're able to achieve such high efficiency.
If governments wanted to, they could mandate that datacenters use air conditioning instead of evaporative cooling, and water usage would drop to near zero (just enough for the restrooms, watering the plants, etc). But nobody would ever seriously suggest doing this because it would be using more of a valuable resource (electricity / CO2 emissions) to save a small amount of a cheap and relatively plentiful resource (water).
> The water issue really is a distraction which harms the credibility of people who lean on it
Is that really the case? - "Data Centers and Water Consumption" - https://www.eesi.org/articles/view/data-centers-and-water-co...
"...Large data centers can consume up to 5 million gallons per day, equivalent to the water use of a town populated by 10,000 to 50,000 people..."
"I Was Wrong About Data Center Water Consumption" - https://www.construction-physics.com/p/i-was-wrong-about-dat...
"...So to wrap up, I misread the Berkeley Report and significantly underestimated US data center water consumption. If you simply take the Berkeley estimates directly, you get around 628 million gallons of water consumption per day for data centers, much higher than the 66-67 million gallons per day I originally stated..."
Also from that article:
> U.S. data centers consume 449 million gallons of water per day and 163.7 billion gallons annually (as of 2021).
Sounds bad! Now let's compare that to agriculture.
USGS 2015 report: https://pubs.usgs.gov/fs/2018/3035/fs20183035.pdf has irrigation at 118 billion gallons per day - that's 43,070 billion gallons per year.
163.7 billion / 43,070 billion * 100 = 0.38 - less than half a percentage point.
It's very easy to present water numbers in a way that looks bad until you start comparing them thoughtfully.
I think comparing data center water usage to domestic water usage by people living in towns is actually quite misleading. UPDATE: I may be wrong about this, see following comment: https://news.ycombinator.com/item?id=45926469#45927945
Agriculture feeds people, Simon.
It's fair to be critical of how the ag industry uses that water, but a significant fraction of that activity is effectively essential.
If you're going to minimize people's concern like this, at least compare it to discretionary uses we could ~live without.
The data's about 20 years old, but for example https://www.usga.org/content/dam/usga/pdf/Water%20Resource%2... suggests we were using over 2b gallons a day to water golf courses.
The vast majority of water in agriculture goes to satisfy our taste buds, not nourish our bodies. Feed crops like alalfa consume huge amounts of water in the desert southwest but the desert climate makes it a great place to grow and people have an insatiable demand for cattle products.
We could feed the world with far less water consumption if we opted not to eat meat. Instead, we let people make purchasing decisions for themselves. I'm not sure why we should take a different approach when making decisions about compute.
> We could feed the world with far less water consumption if we opted not to eat meat.
If you look at the data for animals, that’s not really true. See [1] especially page 22 but the short of it is that the vast majority of water used for animals is “green water” used for animal feed - that’s rainwater that isn’t captured but goes into the soil. Most of the plants used for animal feed don’t use irrigation agriculture so we’d be saving very little on water consumption if we cut out all animal products [2]. Our water consumption would even get a lot worse because we’d have to replace that protein with tons of irrigated farmland and we’d lose the productivity of essentially all the pastureland that is too marginal to grow anything on (50% of US farmland, 66% globally).
Animal husbandry has been such a successful strategy on a planetary scale because it’s an efficient use of marginal resources no matter how wealthy or industrialized you are. Replacing all those calories with plants that people want to actually eat is going to take more resources, not less, especially when you’re talking about turning pastureland into productive agricultural land.
[1] https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1...
[2] a lot of feed is also distiller’s grains used for ethanol first before feeding them to animals, so we’d wouldn’t even cut out most of that
That paper makes the opposite argument than you thought it made. Even from a freshwater perspective it’s more water-efficient to get calories/protein/fat directly from crops than from animal products.
Since you like their work, the authors of your paper answered that question more generally here https://link.springer.com/article/10.1007/s10021-011-9517-8 where they conclude "The water footprint of any animal product is larger than the water footprint of crop products with equivalent nutritional value".
You make some often debunked claims, like we'd have to plant more crops to feed humans directly if we stopped eating meat.
This shouldn't make intuitive sense to you since animals eat feed grown on good cropland (98% of the water footprint of animal ag) that we could eat directly, and we lose 95% of the calories when we route crops through animals.
Ask people which they'd rather have: -no more meat and a little better AI -keep their meat and AI doesn't improve from where it is today..
I mean it's even simpler. Almonds are entirely non essential (many other more water efficient nuts) to the food supply and in California consume more water than the entire industrial sector, and a bit more than all residential usage (~5 million acre-feet of water).
Add a datacenter tax of 3x to water sold to datacenters and use it to improve water infrastructure all around. Water is absolutely a non-issue medium term, and is only a short term issue because we've forgotten how to modestly grow infrastructure in response to rapid changes in demand.
Thanks for the sanity!! I wish more people understood this
I called out golf in my first comment in this thread.
If data center usage meant we didn't have enough water for agriculture I would shout that from the rooftops.
Yep--I'm agreeing that one's a good comparison to elaborate on.
Exploring how it stacks up against an essential use probably won't persuade people who perceive it as wasteful.
Growing almonds is just as essential as building an AI. Eating beef at the rate americans do is not essential. Thats where basically all the water usage is going.
Agriculture is generally essential but that doesn't mean that any specific thing done in the name of agriculture is essential.
If Americans cut their meat consumption by 10%, we would use a lot less water in agriculture and probably also live longer in general
Iran's ongoing water crisis is an example. One cause of it is unnecessary water-intensive crops that they could have imported or done without (just consume substitutes).
It's a common reasoning error to bundle up many heterogeneous things into a single label ("agriculture!") and then assign value to the label itself.
I am surprised by your analytical mistake of comparing irrigation water with data-center water usage...
They are not equivalent. Data centers primarily consume potable water, whereas irrigation uses non-potable or agricultural-grade water. Mixing the two leads to misleading conclusions on the impact.
That's a really good point - you're right, comparing data center usage to potable water usage by towns is a different and more valid comparison than comparing with water for irrigation.
They made a good point, but keep in mind that they're doing a "rules for thee, not for me" sometimes.
The same person who mentioned potable water being an important distinction also cited a report on data center water consumption that did not make the distinction (where the 628M number came from).
The problem is many data centers are in areas where water systems are supply constrained... - https://spectrum.ieee.org/ai-water-usage
This is not a distinction that your second link (that has the 628M number) was making either
> water evaporation from hydroelectric dam reservoirs in their water use calculations
The factual soundness of my argument is independent of the report quality :-) the report influences comprehension, not correctness...
The fact data centers are already having a major impact on the public water supply systems is known, by the decisions some local governments are forced to do, if you care to investigate...
https://spectrum.ieee.org/ai-water-usage
"...in some regions where data centers are concentrated—and especially in regions already facing shortages—the strain on local water systems can be significant. Bloomberg News reports that about two-thirds of U.S. data centers built since 2022 are in high water-stress areas.
In Newton County, Georgia, some proposed data centers have reportedly requested more water per day than the entire county uses daily. Officials there now face tough choices: reject new projects, require alternative water-efficient cooling systems, invest in costly infrastructure upgrades, or risk imposing water rationing on residents...."
https://www.bloomberg.com/graphics/2025-ai-impacts-data-cent...
What counts as data center water consumption here? There are many ways to arguably come up with a number.
Does it count water use for cooling only, or does it include use for the infrastructure that keeps it running (power generation, maintenance, staff use, etc.)
Is this water evaporated? Or moved from A to B and raised a few degrees.
This is the real point. Just measuring the amount of water involved makes no sense. Taking 100 liters of water from a river to cool a plant and dumping them back in a river a few degrees warmer is different from taking 100 liters from a fossil acquifer to evaporatively cool the same plant.
Humans don’t consume much water used by humans. Cows do.
A farmer is a valuable perspective but imagine asking a lumberjack about the ecological effects of deforestation, he might know more about it than an average Joe, but there's probably better people to ask for expertise?
> Honestly, we probably use more water than they do
This kind of proves my point, regardless of the actual truth in this regard, it's a terrible argument to make: availability of water starts to become a huge problem in a growing amount of places, and this statement implies the water usage of something, that in basic principle doesn't need water at all, uses comparable amount of water as farming, which strictly relies on water.
The author of the article followed the quote from the farmer with a fact-checked (this is the New Yorker) note about water usage for AI training.
I think the point here is that objecting to AI data center water use and not to say, alfalfa farming in Arizona, reads as reactive rather than principled. But more importantly, there are vast, imminent social harms from AI that get crowded out by water use discourse. IMO, the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology.
> the environmental attack on AI is more a hangover from crypto than a thoughtful attempt to evaluate the costs and benefits of this new technology
Especially since so many anti-crypto people immediately pivoted to anti-AI. That sudden shift in priorities makes it hard to take them seriously.
I didn't pivot - I'm still strongly anti-crypto and pretty strongly anti-AI.
But if I say "I object to AI because <list of harms> and its water use", why would you assume that I don't also object to alfalfa farming in Arizona?
Similarly, if I say "I object to the genocide in Gaza", would you assume that I don't also object to the Uyghur genocide?
This is nothing but whataboutism.
People are allowed to talk about the bad things AI does without adding a 3-page disclaimer explaining that they understand all the other bad things happening in the world at the same time.
No, that's not the point.
If you take a strong argument and through in an extra weak point, that just makes the whole argument less persuasive (even if that's not rational, it's how people think).
You wouldn't say the "Uyghur genocide is bad because of ... also the disposable plastic crap that those slave factories produce is terrible for the environment."
Plastic waste is bad but it's on such a different level from genocide that it's a terrible argument to make.
Adding a weak argument is a red flag for BS detectors. It's what prosecutors do to hoodwink a jury into stacking charges over a singular underlying crime.
Because your argument is more persuasive to more people if you don't expand your criticism to encompass things that are already normalized. Focus on the unique harms IMO.
I don't think there's a world where a water use tax is levied such that 1) it's enough for datacenters to notice and 2) doesn't immediately bankrupt all golf courses and beef production, because the water use of datacenters is just so much smaller.
We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Beef, I guess, is a popular type of food. I’m under the impression that most of us would be better off eating less meat, maybe we could tax water until beef became a special occasion meal.
I'm saying that if you taxed water enough for datacenters to notice, beef would probably become uneconomical to produce at all. Maybe a good idea! But the reason datacenters would keep operating and beef production wouldn't is that datacenters produce way more utility per gallon.
If it only applies to beef, fine. If it's ALL agriculture...
I can live without another datacenter - I get very little utility from "one more" - but I have to eat, generally every day..
Taxes can have nuance.
You can easily write a law that looks like this: There is now a water usage tax. It applies only to water used for data-centers. It does not apply to residential use, agricultural use, or any other industrial use.
We do preferential pricing and taxing all the time. My home's power rate through the state owned utility is very different than if I consumed the exact same amount of power, but was an industrial site. I just checked and my water rate at home is also different than if I were running a datacenter. So in all actuality we already discriminate for power and water based on end use. at least where I live. Most places I have lived have different commercial and residential rates.
In other words, the price of beef can stay the same.
>In other words, the price of beef can stay the same.
And yet, if you believe the environmentalist argument in the first place, the price of beef should go up for the damages it causes. Hence why a lot of people think the people complaining about AI are wearing an environmentalist mask, rather than having an actual care about the environment.
Pigouvian taxes are fine, but should be applied across all sources of damage.
A lot of beef is produced in such a way that taxing municipal water won't make a material difference. Even buying market rate water rights in the high desert, which already happens in beef production, is a pretty small tariff on the beef.
> We definitely shouldn’t worry about bankrupting golf courses, they are not really useful in any way that wouldn’t be better served by just having a park or wilderness.
Might as well get rid of all the lawns and football fields while we’re at it.
That's a debate worth having too! And it doesn't block this AI debate from continuing nonetheless.
Water taxes should probably be regional. The price of water in the arid Southwest is much higher than toward the East coast. You might see both datacenters and beef production moving toward states like Tennessee or Kentucky.
You're not wrong.
My perspective from someone who wants to understand this new AI landscape in good faith. The water issue isn't the show stopper it's presented as. It's an externality like you discuss.
And in comparison to other water usage, data centers don't match the doomsday narrative presented. I know when I see it now, I mentally discount or stop reading.
Electricity though seems to be real, at least for the area I'm in. I spent some time with ChatGPT last weekend working to model an apples:apples comparison and my area has seen a +48% increase in electric prices from 2023-2025. I modeled a typical 1,000kWh/month usage to see what that looked like in dollar terms and it's an extra $30-40/month.
Is it data centers? Partly yes, straight from the utility co's mouth: "sharply higher demand projections—driven largely by anticipated data center growth"
With FAANG money, that's immaterial. But for those who aren't, that's just one more thing that costs more today than it did yesterday.
Coming full circle, for me being concerned with AI's actual impact on the world, engaging with the facts and understanding them within competing narratives is helpful.
Not only electricity, air pollution around some datacenters too
https://www.politico.com/news/2025/05/06/elon-musk-xai-memph...
I'd love to say the air pollution issue get the attention that's currently being diverted to the water issue!
In what way are they not paying for it?
Farmland, AI data centers, and golf courses do not provide the same utility for water used. You are not making an argument against the water usage problem, you are only dismissing it.
Growing almonds uses 1.3 trillion gallons of water annually in California alone.
This is more than 4 times more than all data centers in the US combined, counting both cooling and the water used for generating their electricity.
What has more utility: Californian almonds, or all IT infrastructure in the US times 4?
Almonds are pretty cherry picked here as notorious for their high water use. Notably, we're not betting an entire economy and pouring increasing resources into almond production, either. Your example would be even more extreme if you chose crops like the corn used to feed cattle. Feeding cows alone requires 21.2 trillion gallons per year in the US.
The people advocating for sustainable usage of natural resources have already been comparing the utility of different types of agriculture for years.
Comparatively, tofu is efficient to produce in terms of land use, greenhouse gas emissions, and water use, and can be made shelf-stable.
>Notably, we're not betting an entire economy and pouring increasing resources into almond production,
Excellent, that means we can save massive amounts of water by stopping almond production in the western US.
People have been sounding the alarm about excessive water diverted to almond farming for many years though, so that doesn't really help the counter-argument.
Example article from a decade ago: https://www.motherjones.com/environment/2015/01/almonds-nuts...
Depends on what the datacenters are used for.
AI has no utility.
Almonds make marzipan.
AI has way more utility than you are claiming and less utility than Sam Altman and the market would like us to believe. It’s okay to have a nuanced take.
"AI has no utility" is a pretty wild claim to make in the tail end of 2025.
Still surprised to see so many take this as such a hot claim. Is there hype, absolutely, is there value being driven also absolutely.
Whenever I see someone say AI has no utility, I'm happy that I don't have to waste time in an argument against someone out of touch with reality.
I'm more unhappy than happy, as there are plenty of points about the very real bad side of AI that are hurt by such delusional and/or disingenuous arguments.
That is, the topic is not one where I have already picked a side that I'd like to win by any means necessary. It's one where I think there are legitimate tradeoffs, and I want the strongest arguments on both sides to be heard so we get the best possible policies in the end.
I agree, but you can't win against religious people. Better spend your time talking to the rest of us.
The article made a really interesting and coherent argument, for example. That's the kind of discourse around the topic I'd like to see.
It's like the smooth brains who still post "lol 6 fingers"
Well, I don't like marzipan, so both are useless? Or maybe different people find uses/utility from different things, what is trash for one person can be life saving for another, or just "better than not having it" (like you and Marzipan it seems).
Marzipan is fun, but useful?
AI is at least as useful as marzipan.
ok in that case you don't need to pick on water in particular, if it has no utility at all then literally any resource use is too much, so why bother insisting that water in particular is a problem? It's pretty energy intensive, eg.
AI has no utility _for you_ because you live in this bubble where you are so rabidly against it you will never allow yourself to acknowledge it has non-zero utility.
Activated almonds create funko pops. I’d still take the data centers over the funko pops buying basedboys that almonds causes.
What does it mean to “use” water? In agriculture and in data centers my understanding is that water will go back to the sky and then rain down again. It’s not gone, so at most we’re losing the energy cost to process that water.
The problem is that you take the water from the ground, and you let it evaporate, and then it returns to... Well to various places, including the ground, but the deeper you take the water from (drinking water can't be taken from the surface, and for technological reasons drinking water is used too) the more time it takes to replenish the aquifer - up to thousands of years!
Of course surface water availability can also be a serious problem.
So with the water used in datacenters. It's just a cooling loop, the output is hot water.
and water from datacenters goes where...? just disappears?
No it’s largely the same situation I think. I was drawing a distinction between agricultural use and maybe some more heavy industrial uses while the water is polluted or otherwise rendered permanently unfit for other uses.
I'll take the almonds any day.
Other people might have other preferences. Maybe we could have a price system where people can express their preferences by paying for things with money, providing more money to the product which is in greater demand?
Sure.. Except some people / companies have so much more money, they can demand impractical things and pay above-market rates for them, causing all others to scramble to live day-to-day with the distorted market.
Tried buying a GPU lately?
Right, I think a data center produces a heck of a lot more economic and human value in a year - for a lot more people - than the same amount of water used for farming or golf.
you can make a strong argument for the greater necessity of farming for survival, but not for golf...
I mean... Food is pretty important ...
Which is why the comparison in the amount of water usage matters.
Data centers in the USA use less than a fraction of a percent of the water that's used for agriculture.
I'll start worrying about competition with water for food production when that value goes up by a multiple of about 1000.
The water intensity of American food production would be substantially less if we gave up on frivolous things like beef, which requires water vastly out of proportion to its utility. If the water numbers for datacenters seem scary then the water use numbers for the average American's beef consumption is apocalyptic.
I appreciate that you feel this way, it’ll never happen.
The US will never give up on eating meat. Full stop.
For every vegan/vegetarian in the US there are probably 25 people that feed beef products to their pets on a daily basis.
While I agree, the "meat is not sustainable" argument is literal, and evidenced in beef prices rising as beef consumption lowers over the past years. Beef is moving along the spectrum from having had been a "staple" to increasingly being a luxury.
The US never gave up eating lobster either, but many here have never had lobster and almost nobody has lobster even once a week. It's a luxury which used to be a staple.
Beef is not the only meat. Chicken is much less water intensive.
From the animal welfare perspective, there's much more suffering involved in producing a pound of chicken than a pound of beef.
That depends how sentient a chicken is: their brains are of similar complexity to the larger of these models, counting params as synapses.
Also, while I'm vegetarian going on vegan, welfare arguments are obviously not relevant in response to an assertation that Americans aren't going to give up meat, because if animal welfare was relevant then Americans would give up meat.
Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds.
Also a lot less meat in general. A huge part of our agriculture is feed to feed our food. We need some meat, but the current amount is excessive
> Corn, potatoes and wheat are important maybe even oranges, but we could live with a lot less alfalfa and almonds. Both alfalfa and almonds contain a lot of nutrients you dont find in large enough amounts (or at all) in corn and potatoes though. And alfalfa improves the soil but fixating nitrogen. Sure almonds require large amounts of water. Maybe alfalfa does as well? And of course it depends on if they are grown for human consumption or animal.
Water usage largely depends on the context, if the water source is sustainable, and if it is freshwater.
Of course water used up will eventually evaporate, and produce rainfall in the water cycle, but unfortunately at many places "fossil" water is used up, or more water used in an area then the watershed can sustainably support.
This is a constant source of miscommunication about water usage, and that of agriculture also. It is very different to talk about the water needs to raise a cow in eg. Colorado and in Scotish highlands, but this is usually removed from the picture.
The same context should be considered for datacenters.
They are making an anti-disruption argument.
I think it's bad though to be against growth, for reasons I've described in another comment.
who are you to determine the utility? we have the market for it and it has spoken.
That is correct, AI data centers deliver far more utility per unit of water than farm/golf land.
Another issue is that you could, in principle, build data centers in places where you don't need to evaporate water to cool them. For example, you could use a closed loop water cooling system and then sink that heat into the air or into a nearby body of water. OVH's datacenter outside Montreal¹ does this, for example. You can also use low carbon energy sources to power the data center (nuclear or hydro are probably the best because their production is reliable and predictable).
Unlike most datacenters, AI datacenters being far away from the user is okay since it takes on the order of seconds to minutes for the code to run and generate a response. So, a few hundred milliseconds of latency is much more tolerable. For this reason, I think that we should pick a small number of ideal locations that have a combination of weather that permits non-sub-ambient cooling and have usable low carbon resources (either hydropower is available and plentiful, or you can build or otherwise access nuclear reactors there), and then put the bulk of this new boom there.
If you pick a place with both population and a cold climate, you could even look into using the data center's waste heat for district heating to get a small new revenue stream and offset some environmental impact.
1: https://www.youtube.com/watch?v=RFzirpvTiOo
I think the water use arguments are relevant, particularly in regions of the world and US (CA) where potable water is scarce, but land and electricity are available .
NYT article gift link where people reported wells ran dry after data centers moved in. : 'From Mexico to Ireland, Fury Mounts Over a Global A.I. Frenzy' https://www.nytimes.com/2025/10/20/technology/ai-data-center...
From https://www.eesi.org/articles/view/data-centers-and-water-co... , I understand there are two types of cooling with water in DCs, open-loop that's simple but water-intensive, and closed-loop that's expensive but efficient.
>> This can be achieved through air cooling using water evaporation, which is an open-loop and more water-intensive method, or through server liquid cooling.
This article goes into an ungodly amount of detail -- with receipts -- making the case that the AI water issue is fake:
https://andymasley.substack.com/p/the-ai-water-issue-is-fake
One of the biggest problems in environmental education (imo) is the lack of notion regarding the footprint of products and services we consume daily; from the water and CO2 costs of a meal, of a heated pool, of a car, etc. It is eye-opening.
I first came across this type of info with the book "How Bad Are Bananas", from Mike Berners-Lee. I really enjoyed it, and I just saw that the new edition even includes stuff like hosting a World Cup, data centers, and space tourism!
It should give a good foundation to start talking about it.
Water location matters. Is the data center in a desert with scarce potable water for locals? Or is next to a large Canadian lake, plenty of potable water, with people who want to trade something for currency so they can put avocados in their salad?
Some time ago, I read the environmental impact assessment for a proposed natural gas thermal power plant, and in it they emphasized that their water usage was very low (to the point that it fit within the unused part of the water usage allowance for an already existing natural gas thermal power plant on the same site) because they used non-evaporative cooling.
What prevents data centers from using non-evaporative cooling to keep their water usage low? The water usage argument loses a lot of its relevant in that case.
Does it route the hot water back into a river?
In europe several power plants get shut down each summer because the heated water from those plants would have significant impact on the local wildlife.
> Does it route the hot water back into a river?
That particular one routed the hot water to a set of fan-cooled radiators (rejecting most of the heat into the air).
It's disheartening that a potentially worthwhile discussion — should we invest engineering resources in LLMs as a normal technology rather than as a millenarian fantasy? — has been hijacked by a (at this writing) 177-comment discussion on a small component of the author's argument. The author's argument is an important one that hardly hinges at all on water usage specifically, given the vast human and financial capital invested in LLM buildout so far.
> but if you compare data centers to farmland or even golf courses they're minuscule.
People are critical of farmland and golf courses, too. But Farmland at least has more benefit for society, so they are more vocal on how it's used.
The problem is more one of scale: a million liters of water is less than half of a single Olympic-sized swimming pool. A single acre of alfalfa typically requires 4.9 - 7.6 million liters a year for irrigation. Also, it's pretty easy to recycle the data center water, since it just has to cool and be sent back, but the irrigation water is lost to transpiration and the recycling-by-weather process.
So, even if there's no recycling, a data center that is said to consume "millions" rather than tens or hundreds of millions is probably using less than 5 acres of alfalfa in consumption, and in absolute terms, this requires only a swimming-pool or two of water per years. It's trivial.
> The problem is more one of scale:
I think the source is the bigger problem. If they take the water from sources which are already scarce, the impact will be harsh. There probably wouldn't be any complaints if they would use sewerage or saltwater from the ocean.
> Also, it's pretty easy to recycle the data center water, since it just has to cool
Cooling and returning the water is not always that simple. I don't know specifically about datacentres, but I know about wasting clean water in other areas, cooling in power plants, industry, etc. and there it can have a significant impact on the cycle. At the end it's a resource which is used at least temporary, which has impact on the whole system.
> If they take the water from sources which are already scarce, the impact will be harsh.
Surprised I had to scroll down this far to see this mentioned.
The water use argument is highly local and depends on where we are building these data centers. Are you building in the great lakes region with plenty of fresh water and no water scarcity issues (yet)? Ok fine.
But we aren't building there. We're building in Arizona, Nevada, Nebraska, Iowa putting further stress in an area that water scarcity is already an issue, or soon going to become one due to long term drought conditions. Or Texas, which already has problems with their power grid.
We're building in these locations because they're cheap. If we're going do to this, we need to not let the bottom line be the sole driving decision of data center locations. If it's not profitable to build elsewhere, don't build it until you've figured out how to make it efficient enough to where it is profitable.
On the water front, in my area agriculture flood irrigates with pristine aquifer water, while surface water gets treated and dumped into the drinking supply. This is due to the economics of historic water rights.
Yes - and the water used is largely non-consumptive.
Not really. The majority of data center water withdrawal (total water input) is consumed ("lost" to evaporation etc...) with a minority of it discharged (returned in liquid form). I believe it's on the order of 3/4ths consumed, but that varies a lot by local climate and cooling technology.
There's lots of promising lower-consumption cooling options, but seems like we are not yet seeing that in a large fraction of data centers globally.
Water and power are local issues. And data center use of water and power is already, currently having local impact on politics. I saw ads about it during the election cycle that just concluded. Candidates had to answer questions about it at debates and in interviews.
People are using these arguments for the simple reason that they demonstrably resonate with average people who live near data centers.
They probably don’t resonate with people who have plenty of income and/or do not compete with data centers locally for resources.
The nice thing about the data center water usage panic is that whenever someone appeals to it, I immediately know that either they haven't done their homework or they're arguing in bad faith.
> data centers often use potable drinking water
hmm why exactly? mineral content?
Yeah I think it's to avoid mineral buildup on the cooling equipment which would then need to be replaced more often.
Water can range from serious concern to NBD depending on where the data center is located, where the water is coming from, and the specific details of how the data center's cooling systems are built.
To say that it's never an issue is disingenuous.
Additionally one could image a data center built in a place with a surplus of generating capacity. But in most cases, it has a big impact on the local grid or a big impact on air quality if they bring in a bunch of gas turbines.
I suppose instead we can talk about people's 401k's being risked in a market propped up by the AI bubble.
We really need to stop tying people's retirement to the market. I've already lost ground due to 2008, and COVID, and while I was young, I lived through my parents suffering through dotcom as well.
It's long past time we have a universal public pension, funded at least in part with a progressive wealth tax, or least go back to private defined benefit pensions to where the employer or pension fund bears the risk rather than the individual.
Supplement it with market speculation if you want, but we need something guaranteed for everyone that's a little more robust and provides a better living than the paltry social security we have now.
Absolutely.
The water argument rings a bit hollow for me not due to whataboutism but more that there's an assumption that I know what "using" water means, which I am not sure I do. I suspect many people have even less of an idea than I do so we're all kind of guessing and therefore going to guess in ways favorable to our initial position whatever that is.
Perhaps this is the point, maybe the political math is that more people than not will assume that using water means it's not available for others, or somehow destroyed, or polluted, or whatever. AFAIK they use it for cooling so it's basically thermal pollution which TBH doesn't trigger me the same way that chemical pollution would. I don't want 80c water sterilizing my local ecosystem, but I would guess that warmer, untreated water could still be used for farming and irrigation. Maybe I'm wrong, so if the water angle is a bigger deal than it seems then some education is in order.
If water is just used for cooling, and the output is hotter water, then it's not really "used" at all. Maybe it needs to be cooled to ambient and filtered before someone can use it, but it's still there.
If it was being used for evaporative cooling then the argument would be stronger. But I don't think it is - not least because most data centres don't have massive evaporative cooling towers.
Even then, whether we consider it a bad thing or not depends on the location. If the data centre was located in an area with lots of water, it's not some great loss that it's being evaporated. If it's located in a desert then it obviously is.
If it was evaporative, the amounts would be much less.
Imnot sure why you're saying it would be less? All sources I can find say that evaporative cooling is a tradeoff of more water for less power.
Just from a physics standpoint.
If you discharge water into a river, there are environmental limits to the outlet temperature (this is a good thing btw). The water can't be very hot. That means you need to pump a large volume of water through because you can only put a small amount of energy into each kg of water.
If you evaporate the water on the other hand, not only is there no temperature limit but it also absorbs the latent heat of vaporisation. The downside is it's a lot more complex and also the water is truly consumed rather than just warming it up.
Put that way, any electricity usage will have some "water usage" as power plants turn up their output (and the cooling pumps) slightly. And that's not even mentioning hydroelectric plants!
I did some napkin math on data center water usage for a 500MW data center in the Columbia River valley.
It uses as much water per year as 200 acres of alfalfa in California’s Central Valley. There are around 1M acres of alfalfa growing in California.
2.5MW of data center capacity is roughly equal to 1 acre of irrigated alfalfa in water usage. If you’re pulling fossil aquifer water, open loop evaporative cooling may not be the best idea, but there are plenty of places east of 100 degrees west in the US that have virtually ‘unlimited’ water where cooling towers are a great idea since they almost double the COP of a chilled water system.
I went down that “water use” rabbit hole a month ago and basically… it’s just a bunch of circular reporting that was based on some paper published in 2023[1]. For ChatGPT 3.5 they claimed “500ml for 10-50 responses”. In 2024, Washington Post published an article that took their claim and said “519 milliliters per email”[2] but didn’t source that from the original paper… that “shocking headline” took off and got widely circulated and cited directly, treating the WaPo calculation as if it were the original research finding. Then tech blogs and advocacy sites ran with it even harder, citing each other instead of any actual research[3].
If you look at the original paper they are quite upfront with the difficulty of estimating water use. It’s not public data—in fact it’s usually a closely held trade secret, plus it’s got all kinds of other issues like you don’t know where the training happened, when it happened, what the actual cooling efficiency was, etc. The researchers were pretty clear about these limitations in the actual paper.
Basically, it’s urban legend at this point. When OpenAI’s CEO later said ChatGPT uses ~0.3ml per query, that’s roughly 100x less than the viral claims.
[1] <https://arxiv.org/abs/2304.03271> [2] <https://www.washingtonpost.com/technology/2024/09/18/energy-...> [3] <https://www.seangoedecke.com/water-impact-of-ai>/
Your context is a little lacking. Golf courses almost universally have retention ponds/wells/etc at the facility and recycle their water.
Only 14% use municipal water systems to draw water. https://www.usga.org/content/dam/usga/pdf/Water%20Resource%2...
"Presented by the USGA" (the United States Golf Association) gave me a wry chuckle there.
That said, here are the relevant numbers from that 2012 article in full:
> Most 18-hole golf facilities utilize surface waters (ponds, lakes) or on-site irrigation wells. Approximately 14 percent of golf facilities use water from a public municipal source and approximately 12 percent use recycled water as a source for irrigation.
> Specific water sources for 18-hole courses as indicated by participants are noted below:
> 52 percent use water from ponds or lakes.
> 46 percent use water from on-site wells.
> 17 percent use water from rivers, streams and creeks.
> 14 percent use water from municipal water systems.
> 12 percent use recycled water for irrigation.
> sound scary when presented without context
It's not about it being scary, its about it being a gigantic, stupid waste of water, and for what? So that lazy executives and managers can generate their shitty emails they used to have their comms person write for them, so that students can cheat on their homework, or so degens can generate a video of MLK dancing to rap? Because thats the majority of the common usage at this point and creating the demand for all these datacenters. If it was just for us devs and researchers, you wouldn't need this many.
Whether it's a "gigantic" waste of water depends on what those figures mean. It's very important to understand if 25 million liters of water per year is a gigantic number or not.
For comparison it's about 10 olympic-sized swimming pools worth of water, doesn't seem very significant to me. Unless you're going to tell people they're not allowed swimming pools any more because swimming doesn't produce enough utility?
And at any rate, water doesn't get used up! It evaporates and returns to the sky to rain down again somewhere else, it's the most renewable resource in the entire world.
If only millions of people suffering from lack of water knew this.
If you redistributed this water to a million people suffering from lack of water, they'ed get about 2 shot glasses worth per day.
Would we be sending that water to those millions of people instead?
Seems the problem is the revealed preference of the normies, rather than the technology itself.
And also, none of those current use cases are a real benefit to society, outside of maybe research cases.
The only benefit is to the already wealthy owner class that is itching to not have to pay for employees anymore because it impacts their bottom line (payroll is typically the largest expense).
It's not like we are making robots to automate agriculture and manufacturing to move toward a post scarcity, moneyless society, which would have real benefits. No, instead we have AI companies hyping up a product whose purpose (according to them) is so that already wealthy people can hoard more wealth and not have to pay for employees. It's promising to take away a large portion of the only high-paying jobs we have left for the average person without an advanced degree.
Me being able to write software a little faster, without hiring a junior, is a net negative to society rather than a benefit.
You appear to be arguing against using technology to boost human efficiency on a forum full of software engineers who've dedicated their careers to building software that makes humans more efficient.
If we aren't doing that then why are we building software?
Because the stated goal of generative AI is not to make an individual more efficient, it's to replace that individual all together and completely eliminate the bottom rungs of the professional career ladder.
Historically software that made humans more efficient resulted in empowerment for the individual, and also created a need for new skilled roles. Efficiency gains were reinvested into the labor market. More people could enter into higher paying work.
With generative AI, if these companies achieve their stated goals, what happens to the wealth generated by the efficiency?
If we automate agriculture and manufacturing, the gain is distributed as post-scarciaty wealth to everyone.
If we automate the last few remaining white-collar jobs that pay a living wage, the gain is captured entirely by the capital owners & investors via elimination of payroll, while society only loses one of its last high-paying ladders for upward mobility.
Nobody lost their career because we built a faster operating system or a better compiler. With generative AI's stated goals, any efficiency gains are exclusively for those at the very top, while everyone else gets screwed.
Now, I'll concede and say, that's not the AI companies' fault. I'm not saying we shouldn't magically stop developing this technology, but we absolutely need our governments to start thinking about the ramifications it can have and start seriously considering things like UBI to be prepared for when the bottom falls out of the labor market.
Thanks, that's a well argued comment.
I'm not a fan of of the "replace workers with AI" thing myself - I'm much more excited about AI as augmentation for existing workers so they can take on more challenging tasks.
Does the future productivity growth that would have been gained later (due to more junior engineers not entering the field) outweigh the AI gains?
If it's just the little productivity boost now, I think it's a net negative if hiring trends continue.
I think it's a discussion to be had but talent pool is a tragedy of the commons situation.
Its not gigantic and its not a waste. Brainrot creates massive economic value that can be used to pay people for products you are more happy to consume.
Arguments in isolation are usually poor. The water usage arguments usually (always?) comes along with a bunch of other arguments, including power consumption, workers rights, consumer protection, public safety, enshittifcation, etc.
When evaluating the economical cost or morality of a thing, (just like when training a machine learning model) the more data you consider the more accurate the result (although just like statistical modelling it is worth to be wary of overfitting).
Fine, fine: get rid of golf courses too.
As for food production; that might be important? IDK, I am not a silicon "intelligence" so what do I know? Also, I have to "eat". Wouldn't it be a wonderful world if we can just replace ourselves, so that agriculture is unnecessary, and we can devote all that water to AGI.
TIL that the true arc of humanity is to replace itself!
See comment here: https://news.ycombinator.com/item?id=45927268
Given the difference in water usage, more data centers does not mean less water for agriculture in any meaningful way.
If you genuinely want to save water you should celebrate any time an acre of farm land is converted into an acre of data center - all the more water for the other farms!
the value of datacenters is dubious. the value of agriculture, less so.
I make use of AI in my farming operation. Now what?
Yeah, but the AI you use in your farming operation is running on that tiny little box behind the buddy seat.
That too, but I, more specifically, meant LLMs operating in someone else's datacenter here.
Once again, the key thing here is to ask how MUCH value we get per liter of water used.
If data centers and farms used the same amount of water we should absolutely be talking about their comparative value to society, and farms would win.
Farms use thousands of times more water than data centers.
Yes, it is worthwhile to ask how much value we get.
And a whole bunch of us are saying we don't see the value in all these datacenters being built and run at full power to do training and inference 24/7, but you just keep ignoring or dismissing that.
It is absolutely possible that generative AI provides some value. That is not the same thing as saying that it provides enough value to justify all of the resources being expended on it.
The fact that the amount of water it uses is a fraction of what is used by agriculture—which is both one of the most important uses humans can put water to, as well as, AIUI, by far the single largest use of water in the world—is not a strong argument that its water usage should be ignored.
it is also a fraction of golf courses which you again ignore. this is just typical "don't do anything!!" ism. there's no argument here.. even if data centres used .00001 millilitre of water you would say the same thing.
Oh, I think golf courses shouldn't exist. They're awful in a number of ways. You want to play golf? VR or minigolf.
But (as I pointed out elsewhere in this discussion [0]) why should I have to mention everything that uses water in a way I think is detrimental in order to be allowed to talk about why I think this thing uses water in a way that is detrimental?
[0] https://news.ycombinator.com/item?id=45927558
Complaining about someone else using an "ism" during a literal "whataboutism" is rich
Once again, you are ignoring my (implied) argument:
Humans NEED food, the output of agriculture. Humans do not NEED any of LLMs' outputs.
Once everyone is fed, then we can talk about water usage for LLMs.
Farms already produce more than enough food to feed everyone (and, indeed, the excess is a feature because food security is really important). The reason not everyone is fed is not due to needing to divide water resources between farms and other uses.
Stop eating beef. With the water saved we can grow enough food for any realistic human population. Ok we solved this one. Or do humans NEED burgers as well? We can already feed all people, any starvation is strictly a political problem not a food existing on the planet problem
Going only by the effective need of humans is a bad argument. A lot of farmers wouldn't survive without subsidies and are not vital to our food supply.
We produce enough food for everyone already, and then waste a huge amount of it. Our food problem isn't about producing more, it's about distributing what we have.
fine so do the same performative activism for
1. netflix
2. gmail
3. hackernews
4. discord
5. gaming
According to this logic the ideal situation is when there are no farms anymore because then each (out of zero) farm gets maximum water.
Eventually people stop building more data centers as food becomes scarce and expensive, and farms become the hot new thing for the stock market, cereal entrepreneurs become the new celebrities and so on. Elon Husk, cereal magnate.
I'm personally excited for when the AGI-nauts start trotting out figures like...
> An H100 on low-carbon grid is only about 1–2% of one US person’s total daily footprint!
The real culprit is humans after all.
Humans have been measuring between human only vs augmented labor for literal centuries.
Frederick Taylor literally invented the process you describe in his “principles of scientific management”
This is the entire focus of the Toyota automation model.
The consistent empirical pattern is:
Machine-only systems outperform humans on narrow, formalizable tasks.
Human-machine hybrid systems outperform both on robustness, yieldjng higher success probability
Good enough?
I was making a joke.
I think the water usage argument can be pertinent depending on the context.
https://www.bbc.com/news/articles/cx2ngz7ep1eo
https://www.theguardian.com/technology/2025/nov/10/data-cent...
https://www.reuters.com/article/technology/feature-in-latin-...
That BBC story is a great example of what I'm talking about here:
> A small data centre using this type of cooling can use around 25.5 million litres of water per year. [...]
> For the fiscal year 2025, [Microsoft's] Querétaro sites used 40 million litres of water, it added.
> That's still a lot of water. And if you look at overall consumption at the biggest data centre owners then the numbers are huge.
That's not credible reporting because it makes no effort at all to help the reader understand the magnitude of those figures.
"40 million litres of water" is NOT "a lot of water". As far as I can tell that's about the same annual water usage as a 24 acre soybean field.
I agree that those numbers can seem huge without proper context.
For me, that BBC story, and the others, illustrates a trend; tech giants installing themselves in ressource-strained areas, while promoting their development as drivers of economic growth.
Yes, a 24 acre soybean field uses a lot of water.
And an average US soybean farm has 312 acres (13x larger than 24 acres): http://www.ers.usda.gov/topics/crops/soybeans-and-oil-crops/...
Which means that in 2025 Microsoft's Querétaro sites used 1/13th of a typical US soybean farm's annual amount of water.
It's a lot of water for AI waifus and videos of Trump shitbombing people who dare oppose him.
It's not a lot of water for AI weather modeling to ensure the soybean crops throughout the country are adequately watered and maximize yield.
>Tip for AI skeptics
Assumptions you are making:
- AI = transformer ANNs
- People sceptical of transformer ANNs directly leading to AGI within any reasonable period are also sceptical of transformer ANNs directly leading to AGI any time in the far future
This kind of generalisations don't help you as the huge number of comments underneath yours likely shows
I don't think anyone who read my comment here misunderstood my usage of the term "AI skeptic" as applying to any form of machine learning as opposed to modern generative AI.
What you think it is and what it is are 2 different things. You are once more making assumptions.
What assumption did I make here?
It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions.
And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
>And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
To be fair, evolution took around 6 million years to evolve the modern human from the closest "ape" ancestor, and over a couple of billion years from single-celled organisms, so you could probably think about the energy expenditure for that trial and error (by evolution) to arrive at our species.
To be fair to the raw capabilities of the semiconductor industry, a 100mm^2 die at 3nm can contain on the order of 1~10 trillion features. I don't know that we are actually that far off in terms of scale. How to arrange these features seems to be the difficult part.
The EDA [0] problem is immune to the bitter lesson. There are certainly specific arrangements of matter that can solve this problem better than a GPU/TPU/CPU can today.
[0] https://en.wikipedia.org/wiki/Electronic_design_automation
On the other hand, a large part of the complexity of human hardware randomly evolved for survival and only recently started playing around in the higher-order intellect game. It could be that we don't need so many neurons just for playing intellectual games in an environment with no natural selection pressure.
Evolution is winning because it's operating at a much lower scale than we are and needs less energy to achieve anything. Coincidentally, our own progress has also been tied to the rate of shrinking of our toys.
Evolution has won so far because it had a four billion year head start. In two hundred years, technology has gone from "this multi-ton machine can do arithmetic operations on large numbers several times faster than a person" to "this box produces a convincing facsimile of human conversation, but it only emulates a trillion neurons and they're not nearly as sophisticated as real ones."
I do think we probably need a new hardware approach to get to the human level, but it does seem like it will happen in a relative blink of an eye compared to how long the brain took.
But we don't even need a human brain. We already have those, they take months to grow, take forever to train, and are forever distracted. Our logic-based processes will keep getting smaller and less power hungry as we figure out how to implement them at even lower scales, and eventually we'll be able to solve problems with the same building blocks as evolution but in intelligent ways, of which LLMs will likely only play a minuscule part of the larger algorithms.
I think current LLMs are trying to poorly emulate several distinct systems.
They're not that great at knowledge (and we're currently wasting most of the neurons on memorizing common crawl, which... have you looked at common crawl?)
They're not that great at determinism (a good solution here is that the LLM writes 10 lines of Python, which then feed back into the LLM. Then the task completes 100% of the time, and much cheaper too).
They're not that great at complex rules (surprisingly good actually, but expensive and flakey). Often we are trying to simulate what are basically 50 lines of Prolog with a trillion params and 50KB of vague English prompts.
I think if we figure out what we're actually trying to do with these things, then we can actually do each of those things properly, and the whole thing is going to work a lot better.
This is a great summary! I've joked with a coworker that while our capabilities can sometimes pale in comparison (such as dealing with massively high-dimensional data), at least we can run on just a few sandwiches per day.
> We would need a new hardware paradigm.
It's not even that. The architecture(s) behind LLMs are nowhere near close that of a brain. The brain has multiple entry-points for different signals and uses different signaling across different parts. A brain of a rodent is much more complex than LLMs are.
LLM 'neurons' are not single input/single output functions. Most 'neurons' are Mat-Vec computations that combine the products of dozens or hundreds of prior weights.
In our lane the only important question to ask is, "Of what value are the tokens these models output?" not "How closely can we emulate an organic bran?"
Regarding the article, I disagree with the thesis that AGI research is a waste. AGI is the moonshot goal. It's what motivated the fairly expensive experiment that produced the GPT models, and we can look at all sorts of other hairbrained goals that ended up making revolutionary changes.
"To simplify it greatly, an LLM neuron is a single input single output function". This is very wrong unless I'm mistaken. A synthetic neuron is multiple input single output.
Ten thousands of extremely complex analog inputs, one output with several thousand of targets that MIGHT receive the output with different timing and quality.
One neuron is ufathomably complex. It‘s offensive to biology to call a cell in a mathematical matrix neuron.
It's even worse than number of input/outputs, number of neurons, efficiency or directional feedback.
The brain also has plasticity! The connections between neurons change dynamically - an extra level of meta.
>It is ultimately a hardware problem.
I think it's more an algorithm problem. I've been reading how LLMs work and the brain does nothing like matrix multiplication over billions of entities. It seems a very inefficient way to do it in terms of compute use, although efficient in terms of not many lines of code. I think the example of the brain shows one could do far better.
Assuming you want to define the goal, "AGI", as something functionally equivalent to part (or all) of the human brain, there are two broad approaches to implement that.
1) Try to build a neuron-level brain simulator - something that is a far distant possibility, not because of compute, but because we don't have a clear enough idea of how the brain is wired, how neurons work, and what level of fidelity is needed to capture all the aspects of neuron dynamics that are functionally relevant rather than just part of a wetware realization
OR
2) Analyze what the brain is doing, to extent possible given our current incomplete knowledge, and/or reduce the definition of "AGI" to a functional level, then design a functional architecture/implementation, rather than neuron level one, to implement it
The compute demands of these two approaches are massively different. It's like the difference between an electronic circuit simulator that works at gate level vs one that works at functional level.
For time being we have no choice other than following the functional approach, since we just don't know enough to build an accurate brain simulator even if that was for some reason to be seen as the preferred approach.
The power efficiency of a brain vs a gigawatt systolic array is certainly dramatic, and it would be great for the planet to close that gap, but it seems we first need to build a working "AGI" or artificial brain (however you want it define the goal) before we optimize it. Research and iteration requires a flexible platform like GPUs. Maybe when we figure it out we can use more of a dataflow brain-like approach to reduce power usage.
OTOH, look at the difference between a single user MOE LLM, and one running in a datacenter simultaneously processing multiple inputs. In the single-user case we conceptualize the MOE as saving FLOPs/power by only having one "expert" active at a time, but in the multi-user case all experts are active all the time handling tokens from different users. The potential of a dataflow approach to save power may be similar, with all parts of the model active at the same time when handling a datacenter load, so a custom hardware realization may not be needed/relevant for power efficiency.
Or
3) Pour enough computation into a sufficiently capable search process and have it find a solution for us
Which is what we're doing now.
The bitter lesson was proven right once again. LLMs prove that you can build incredibly advanced AIs without "understanding" how they work.
You could do an architectural search, and Google previously did that for CNNs with it's NASNet (Network Architectural Search) series of architectures, but the problem is you first need to decide what are the architectural components you want your search process to operate over, so you are baking in a lot of assumptions from the start and massively reducing the search space (because this is necessary to be computationally viable).
A search or evolutionary process would also need an AGI-evaluator to guide the search, and this evaluator would then determine the characteristics of the solution found, so it rather smacks of benchmark gaming rather than the preferred approach of designing for generic capabilities rather than specific evaluations.
I wouldn't say we don't know how LLMs "work" - clearly we know how the transformer itself works, and it was designed intentionally with certain approach in mind - we just don't know all the details of what representations it has learnt from the data. I also wouldn't say LLMs/transformers represent a bitter lesson approach since the architecture is so specific - there is a lot of assumptions baked into it.
Hard problem of consciousness seems way harder to wolve than the easy one which is a purely engineering problem. People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea.
> People have been thinking about why the brain thinks for a very long time and so far we have absolutely no idea
I'm not sure what you mean by this.
I think there is a pretty large consensus that our neocortex is a prediction machine (predicting future observations/outcomes from past experience), and the reason WHY it would have evolved to be this is because there is obvious massive survival benefit in successfully predicting how predators and prey will react ahead of time, what will be the outcome of your own actions, etc, etc. Prediction unlocks you from being stuck in the present having to react to things as they happen and lets you plan ahead.
Thinking = Reasoning/Planning is just multi-step prediction.
I don't think consciousness is the big deal most people think it is - it seems to be just the ability to self-observe (which helps to self-predict), but if we somehow built AGI that wasn't conscious, then who cares?
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
Correct - the vast majority of people vastly underestimate the complexity of the human brain and the emergent properties that develop from this inherent complexity.
it is an architecture problem, too. LLMs simply aren't capable of AGI
Why not?
A lot of people say that, but no one, not a single person has ever pointed out a fundamental limitation that would prevent an LLM from going all the way.
If LLMs have limits, we are yet to find them.
Real time learning that doesn't pollute limited context windows.
We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI. Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic and physical constraints on continuing this trend indefinitely.
OOD is still unsolved problem, they basically struggle under domain shifts and long tail cases or when you try systematically new combinations of concepts (especially on reasoning heavy tasks). This is now a well documented limitation of LLMs/multimodal LLMs.
Work on COT faithfulness shows that the step by step reasoning they print doesn't match their actual internal computation, they frequently generate plausible but misleading explanations of their own answers (lookup anthropic paper). That means they lack self knowledge about how/why they got a result. I doubt you can get AGI without that.
None of this proves that no LLM based architecture could ever reach AGI. But it directly contradicts the idea that we haven't found any limits. We've already found multiple major limitations of the current LLMs, and there's no evidence that blindly scaling this recipe is enough to cross from very capable assistant to AGI.
“And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.”
I’ve always thought about nature didn’t evolve to use electricity as its primary means of energy. Instead it uses chemistry. It’s quite curious, really.
Like a tiny insect is chemistry powered. It doesn’t need to recharge batteries, it needs to eat and breathe oxygen.
What if our computers started to use biology and chemistry as their primary energy source?
Or will it be the case that in the end using electricity as the primary energy source is more efficient for “human brain scale computation”, it’s just that nature didn’t evolve that way…
I remember reading about memristors when I was at University and the hope they could help simulate neurons.
I don't remember hearing much about neuromorphic computing lately though so I guess it hasn't had much progress.
It’s not the level of computing we might hope for, but there has been some progress in developing memristors :)
https://journals.plos.org/plosone/article?id=10.1371/journal...
> To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs
This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
Edit: As for the rest of your argument, it's not so clear cut. An LLM can produce a complete essay in a fraction of the time it would take a human. So yes, a human brain only consumes about 20W but it might take a week to produce the same essay that the LLM can produce in a few seconds.
Also, LLMs can process multiple prompts in parallel and share resources across those prompts, so again, the energy use is not directly comparable in the way you've portrayed.
> This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs.
I think it's more than just scaling, you need to understand the functional details to reproduce those functions (assuming those functions are valuable for the end result as opposed to just the way it had to be done given the medium).
An interesting example of this neuron complexity that was published recently:
As rats/mice (can't remember which) are exposed to new stimuli, the axon terminals of a single neuron do not all transmit a signal when there is an action potential, they transmit in a changing pattern after each action potential and ultimately settle into a more consistent pattern of some transmitting and some not.
IMHO: There is interesting mathematical modeling and transformations going on in the brain that is the secret sauce for our intelligence and it is yet to be figured out. It's not just scaling of LLM's, it's finding the right functions.
Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily. I was making a more general point that a single-I/O function can pretty trivially replicate a multi-I/O function, so the OP's point that "LLM neurons" are single-I/O and bio neurons are multi-I/O doesn't mean much. Estimates of brain complexity have already factored this in, which is why we know we're still a few orders of magnitude away from the number of parameters needed for a human brain in a raw compute sense.
However, the human brain has extra parameters that a pure/distilled general intelligence may not actually need, eg. emotions, some types of perception, balance, and modulation of various biological processes. It's not clear how many of the parameters of the human brain these take up, so maybe we're not as far as we think.
And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical. I think general intelligence will likely have multiple models which achieve similar results, just like there are multiple ways to sort a set of numbers.
I agree with both of you, but scaling isn't feasible with this paradigm. You could need continent-sized hardware to approximate general intelligence with the current paradigm.
> You could need continent-sized hardware to approximate general intelligence with the current paradigm.
I doubt it, if by "current paradigm" you mean the hardware and general execution model, eg. matrix math. Model improvements from progress in algorithms have been outpacing performance improvements from hardware progress for decades. Even if hardware development stopped today, models will continue improving exponentially.
That's my non-expert belief as well. We are trying to brute force an approximation of one aspect of how neurons work at great cost.
so planes that don't flap their wings can't fly
Try explaining to someone who's only ever seen dial-up modems that 4k HDR video streaming is a thing.
Minor correction here. You are correct about hardware being an issue, but the magnitude is much greater. You have a lot more than "thousands" of inputs. In the hand alone you have ~40,000+ tactile corpuscles (sensing regions). And that's just one mode. The eye has ~7 million cones and 80 million rods. There is processing and quantization performed by each of those cells and each of the additional cells those signal, throughout the entire sensory-brain system. The amount of data the human brain processes is many orders of magnitude greater than even our largest exascale computers. We are at least 3 decades from AGI if we need equivalent data processing as the human brain, and that's optimistic.
Like you mention, each individual neuron or synapse includes fully parallel processing capability. With signals conveyed by dozens of different molecules. Each neuron (~86 billion) holds state information in addition to processing. The same is true for each synapse (~600 quadrillion). That is how many ~10 Hz "cores" the human computational system has.
The hubris of the AI community is laughable considering the biological complexity of the human body and brain. If we need anywhere close to the same processing capability, there is no doubt we are multiple massive hardware advances away from AGI.
Exactly why I cringe so hard when AI-bros make arguments equating AI neurons to biological neurons.
There are some tradeoffs in the other direction. Digital neurons can have advantages that biological neurons do not.
For example, if biology had a "choice" I am fairly confident that it would have elected to not have leaky charge carriers or relatively high latency between elements. Roughly 20% of our brain exists simply to slow down and compensate for the other 80%.
I don't know that eliminating these caveats is sufficient to overcome all the downsides, but I also don't think we've tried very hard to build experiments that directly target this kind of thinking. Most of our digital neurons today are of an extremely reductive variety. At a minimum, I think we need recurrence over a time domain. The current paradigm (GPU-bound) is highly allergic to a causal flow of events over time (i.e., branching control flows).
The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
I'm an atheist too. I grew up in the church, rejected it in my teens. The problem with organized religion was the "organized" part -- the centralized, inflexible human authority.
I'm increasingly convinced that spirituality is a vital part of the human experience and we should embrace it, not reject it. If you try to banish basic human impulses, they just resurface in worse, unexpected forms somewhere else.
We all need ways to find deep connection with other humans and the universe around us. We need basic moral principles to operate on. I think most atheists like myself have quietly found this or are in the process of finding this, but it's ok to say it out loud.
For me it means meditation, frugality, and strict guidelines on how I treat others. That's like a religion, I guess. But that's OK. I embrace it. By owning it and naming it, you have mastery over it.
Thanks for saying this. I have a similar situation: " I grew up in the church, rejected it in my teens."
I've come to conclude that the problem isn't "religion" in the abstract, it's that the current institutions are unhealthy and sometimes toxic. IE, it's easier to be an atheist, even in name only, when one realizes that the religion they participated in as a child has more problems than benefits.
I sometimes wonder how people like us could create institutions that replace religions, yet generally agree with our beliefs and needs.
---
BTW: I've seen plenty of toxic behavior from atheists too. Aggressive evangelizing ones' beliefs, (or lack thereof,) IMO is a toxic behavior.
> The problem with organized religion was the "organized" part
What on earth is "unorganized" religion? How is such a concept distinct from the concepts of "worldview" or "ideology"? When people use the term, I assume automatically they're referring to abrahamic faiths—especially in contrast to "atheism".
Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations? Those situations are close in other aspects too: we have a physical system (the sun, the brain), whose functionality we try to replicate technologically.
> Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations?
Yes. And the people talking about "bringing the power of the Sun to Earth" are usually either full of shit or expect so little from their audience that they are effectively talking shit (or are making clickbait, what's the same). (And yes, there's a lot of fraud around fusion too.)
The people doing actual work don't use phrases like that in normal conversation.
the way the pro nuclear crowd talk you might think they are a persecuted religion actually
Or over-the-top proponents of clean intermittent energy sources. Mountains of batteries and lakes of pumped storage for the Sun and Wind god.
You don't even have to go as far as fusion reactors. Nuclear bombs are real, and we know they work.
But surely, anyone who's talking about atomic weapons must be invoking religious imagery and the old myths of divine retribution! They can't be talking about an actual technology capable of vaporizing cities and burning people into the walls as shadows - what a ridiculous, impossible notion would that be! "World War 3" is just a good old end-of-the-world myth, the kind of myth that exists in many religions, but given a new coat of paint.
And Hiroshima and Nagasaki? It's the story of Sodom and Gomorrah, now retold for the new age. You have to be a doomsday cultist to believe that something like this could actually happen!
People always create god, even if they claim not to believe in it. The rise of belief in conspiracy theories is a form of this (imagining an all powerful entity behind every random event), as is the belief in AGI. It's not a totalizing solution to all woes. It's just a way to convince oneself that the world is not random, and is therefore predictable, which makes us feel safer. That, after all, is what we are - prediction machines.
The existential dread from uncertainty is so easily exploited too, and the root cause for many of societies woes. I wonder what the antidote is, or if there is one.
It's just a scam, plain and simple. Some scams can go on for a very long time if you let the scammers run society.
Any technically superior solution needs to have a built in scam otherwise most followers will ignore it and the scammers won't have incentive to prosthelytize, e.g. rusts' safety scam.
I've seen more religious language from AGI skeptics than believers. I kind of think AGI will happen on the basis of being able to think / process data like a human brain which I don't see as unlikely. The skeptics will say AGI is trying to build god and so not happening, but that's a strawman argument really.
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
They're a dead end for whatever their definition of "AGI" is, but still incredibly useful in many areas and not a "dead end" economically.
Well, except for that needing a 40 year bond for a 3 year technology cycle thing.
I figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
"Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
"It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
> I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it.
Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes.
So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
Far more common are ideas that don't work on any scale at all.
If you have something that gives a sticky +5% at 250M scale, you might have an actual winner. Almost all new ML ideas fall well short of that.
By the way downvoting me will not hurt my feelings and I understand why you are doing it, I don't care if you believe me or not. In your position I certainly would think the same thing you are. Its fine. The future will come soon enough without my help.
You're being downvoted for displaying the kind of overconfidence that people consider shameful.
Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
I know how it looks. I shouldn't have said anything- it was pointless.
At any rate someone else will figure out what i know soon or already has.
> Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end.
There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
Me too. But, I worry this “want” may not be realistic/scalable.
Yesterday, I was trying to get some Bluetooth/BLE working on a Raspberry CM 4. I had dabbled with this 9 months ago. And things were making progress then just fine. Suddenly with a new trixie build and who knows what else has changed, I just could not get my little client to open the HCI socket. In about 10 minutes prompt dueling between GPT and Claude, I was able to learn all about rfkill and get to the bottom of things. I’ve worked with Linux for 20+ years, and somehow had missed learning about rfkill in the mix.
I was happy and saddened. I would not have k own where to turn. SO doesn’t get near the traffic it used to and is so bifurcated and policed I don’t even try anymore. I never know whether to look for a mailing list, a forum, a discord, a channel, the newsgroups have all long died away. There is no solidly written chapter in a canonically accepted manual written by tech writers on all things Bluetooth for the Linux Kernel packaged with raspbian. And to pile on, my attention span driven by a constant diet of engagement, makes it harder to have the patience.
It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
Distros do have manuals, they just usually come in the form of user-curated wikis these days. ArchWiki is usually my first stop when I run into a Linux issue, even as a fellow Debian user.
Both https://wiki.archlinux.org/title/Bluetooth and https://wiki.debian.org/BluetoothUser mention rfkill and show you how to troubleshoot.
> It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
This is the real AI risk we should be worried about IMO, at least short term. Information technology has made things vastly more complicated. AI will make it even more incomprehensible. Tax code, engineering, car design, whatever.
It's already happening at my work. I work at big tech and we already have a vast array of overly complicated tools/technical debt no one wants to clean up. There's several initiatives to use AI to prompt an agent, which in turn will find the right tool to use and run the commands.
It's not inconceivable that 10 or 20 years down the road no human will bother trying to understand what's actually going on. Our brains will become weaker and the logic will become vastly more complicated.
Has anyone tried putting the AIs to work cleaning up the technical debt?
Yes, I'm already doing it. But the problem is there's not a lot of incentive from management to do it.
Long term investment in something that can't easily be quantified is a non-starter to management. People will say "thank you for doing that" but those who create new features that drive metrics get promoted.
I see this too. The focus on “believable ROI” makes it very hard to do.
I think LLMs as a replacement for Google, Stack Overflow, etc. is a no brainer. As long as you can get to the source documents when you need them, and train yourself to sniff out hallucinations.
(We already do this constantly in categorizing human generated bullshit information and useful information constantly. So learning to do something similar with LLM output is not necessarily worse, just different.)
What's silly at this point is replacing a human entirely with an LLM. LLMs are still fundamentally unsuited for those tasks, although they may be in the future with some significant break throughs.
The LLMs we have today aren't a fantasy: They're a concrete thing that works.
Just because the people who make them live in a fantasy world, doesn't mean we can't reap the fruits of their labor!
That being said, I suspect a lot of the energy spent on AI training is resulting in unusable slop.
Yeah, using LLMs makes me reconsider the complexity of the software I'm producing and I'm relying on. In a sense LLMs can be a test for the complexity and the fast iteration cycles could yield better solutions than the existing ones
I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor.
Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done.
There are big wins in AI. But those don't pump the bubble once they're solved.
And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
Author here. Indeed - it would be just as fantastical to deny there has been no value from deep learning, transformers, etc.
Yesterday I heard Cory Doctorow talk about a bunch of pro bono lawyers using LLMs to mine paperwork and help exonerate innocent people. Also a big win.
There's good stuff - engineering - that can be done with the underlying tech without the hyperscaling.
Not only whispr, so much of the computer vision area is not as in vogue. I suspect because the truly monumental solutions unlocked are not that accessible to the average person; i.e. industrial manufacturing and robotics at scale.
That's because industrial manufacturing and robotics are failing to bring down costs and make people's lives more affordable.
That's really the only value those technologies provide, so if people aren't seeing costs come down there really is zero value coming from those technologies.
I think that LLM hype is hiding a lot of very real and impactful progress in real world/robot intelligence.
An essay writing machine is cool. A machine that can competently control any robot arm, and make it immediately useful is a world-changing prospect.
Moving and manipulating objects without explicit human coded instructions will absolutely revolutionize so much of our world.
I think a lot of AI wins are going to end up local and free much like whisper.
Maybe it could be a little bit more accurate, it would be nice if it ran a little faster, but ultimately it's 95% complete software that can be free forever.
My guess is very many AI tasks are going to end up this way. In 5-10 years we're all going to be walking around with laptops with 100k cores and 1TB of RAM and an LLM that we talk to and it does stuff for us more or less exactly like Star Trek.
How refreshing to see an AI realistic view on HN these days! As the author said, no one is claiming transformer tech useless, the issue is the relentless drive to claim loudly that transformers will lead to AGI in the next few years and solve all existing problems and that it is worth the current negative damage to society and the environment.
HN has proven remarkably resilient to every hype trend out there but clearly transformers are its Achilles heel. That or/and massive transformer astroturfing
The idea of replicating a consciousness/intelligence in a computer seems to fall apart even under materialist/atheist assumptions: what we experience as consciousness is a product of a vast number of biological systems, not just neurons firing or words spoken/thought. Even considering something as basic as how fundamental bodily movement is to mental development, or how hormones influence mood ultimately influencing thought, how could anyone ever hope to to replicate such things via software in a way that "clicks" to add up to consciousness?
Conflating consciousness and intelligence is going to hopelessly confuse any attempt to understand if or when a machine might achieve either.
(I think there's no reasonable definition of intelligence under which LLMs don't possess some, setting aside arguments about quantity. Whether they have or in principle could have any form of consciousness is much more mysterious -- how would we tell?)
Defining machine consciousness is indeed mysterious, at the end of the day it ultimately depends on how much faith one puts in science fiction rather than an objective measure.
Seems like a philosophy question, with maybe some input from neuroscience and ML interpretability. I'm not sure what faith in science fiction has to do with it.
Bundling up consciousness with intelligence is a big assumption, as is the assumption that panpsychism is incorrect. You may be right on both counts, but you can't just make those two assumptions as a foregone conclusion.
I don't see a strong argument here. Are you saying there is a level of complexity involved in biological systems that can not be simulated? And if so, who says sufficient approximations and abstractions aren't enough to simulate the emergent behavior of said systems?
We can simulate weather (poorly) without modeling every hydrogen atom interaction.
The argument is about causation or generation, not simulation. Of course we can simulate just about anything, I could write a program that just prints "Hello, I'm a conscious being!" instead of "Hello, World!".
The weather example is a good one: you can run a program that simulates the weather in the same way my program above (and LLMs in general) simulate consciousness, but no one would say the program is _causing_ weather in any sense.
Of course, it's entirely possible that more and more people will be convinced AI is generating consciousness, especially when tricks like voice or video chat with the models are employed, but that doesn't mean that the machine is actually conscious in the same way a human body empirically already is.
If you simulate rainy weather, does anything get wet?
(Not my original quote, but can't remember right now where I read it.)
It's similar asking about whether silicon computers performing intelligent tasks is "conscious".
I guess it depends, can you tell the difference between a weather simulation and the actual world?
Can you?
You have weather readouts. One set is from a weather simulation - a simulated planet with simulated climate. Another is real recordings from the same place at the same planet, taken by real weather monitoring probes. They have the same starting point, but diverge over time.
Which one is real though? Would you be able tell?
They're not asking about telling the difference in collected data sets, data sets aren't weather.
The question is can you tell the difference between the rain you see outside your window, and some representation of a simulated environment where the computer says "It's raining here in this simulated environment". The implied answer is of course, one is water falling from the sky and one is a machine.
You can't look at the "real weather" though. You can only look at the outputs. That's the constraint. Good luck and have fun.
A human brain is a big pile of jellied meat spread. An LLM is a big pile of weights strung together by matrix math. Neither looks "intelligent". Neither is interpretable. The most reliable way we have to compare the two is by comparing the outputs.
You can't drill a hole in one of those and see something that makes you go "oh, it's this one, this one is the Real Intelligence, the other is fake". No easy out for you. You'll have to do it the hard way.
Even granting all of your unfounded assertions; "the output" of one is the rain you see outside, "the output" of the other is a series of notches on a hard drive (or the SSD equivalent, or something in RAM, etc.) that's then represented by pixels on a screen.
The difference between those two things (water and a computer) is plain, unless we want to depart into the territory of questioning whether that perception is accurate (after all, what "output" led us to believe that "jellied meat spread" can really "perceive" anything?), but then "the output" ceases to be any kind of meaningful measure at all.
there is no "real weather". the rain is the weather. the map is not the territory. these are very simple concepts, idk why we need to reevaluate them because we all of a sudden got really good at text synthesis
Everyone's a practical empiricist until our cherished science fiction worldview is called into question, then all of a sudden it's radical skepticism and "How can anyone really know anything, man?"
You experience everything through digital signals. I dont see why those same signals cant be simulated. You are only experiencing the signal your skin sends to tell you there is rain, you dont actually need skin to experience that signal.
AGI won't replicate our experience.
But it could be more powerful than us.
We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible.
I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
> I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty.
Oh no.
AI data centers are sucking up so much power it's making everyone's electric bill go up.
That's a tangible problem that dramatically impacts the poor and average person.
In addition to being hard, we should ask if something is useful or if the benefits outweigh the harms.
It's hard to see benefits from AI systems, AGI or otherwise. It doesn't seem to produce anything that improves human happiness or general well being.
I think you just lack imagination. I can imagine a lot of benefits of AGI or near-AGI systems.
I agree. The benefits are just not well distributed.
Greedy managers are a blocker to actual engineering. It wasn't enough that they were trying to squeeze the last ounce of delivery via twisted implementations of Agile. Now they are shooting down every attempt to apply any amount of introspection and thought with blanket expectation of LLMs obviating any need to do so. That combined with random regurgitation of terms like "MCP" and "agentic" has made programming into a zombie-like experience of trying to coax the LLMs to produce something workable while fighting inflated expectations of the hallucinating bosses.
On the other hand we have DeepMind / Demis Hassabis, delivering:
* AlphaFold - SotA protein folding
* AlphaEvolve + other stuff accelerating research mathematics: https://arxiv.org/abs/2511.02864
* "An AI system to help scientists write expert-level empirical software" - demonstrating SotA results for many kinds of scientific software
So what's the "fantasy" here, the actual lab delivering results or a sob story about "data workers" and water?
I believe AlphaFold, AlphaEvolve etc are _not_ looking to get to AGI. The whole article is a case against AGI chasing, not ML or LLM overall.
AlphaEvolve is a general system which works in many domains. How is that not a step towards general intelligence?
And it is effectively a loop around LLM.
But my point is that we have evidence that Demis Hassabis knows his shit. Just doubting him on a general vibe is not smart
AlphaEvolve is a system for evolving symbolic computer programs.
Not everything that DeepMind works on (such as AlphaGo, AlphaFold) are directly, or even indirectly, part of a push towards AGI. They seem to genuinely want to accelerate scientific research, and for Hassabis personally this seems to be his primary goal, and might have remained his only goal if Google hadn't merged Google Brain with DeepMind and forced more of a product/profit focus.
DeepMind do appear to be defining, and approaching, "AGI" differently that the rest of the pack who are LLM-scaling true believers, but exactly what their vision is for an AGI architecture, at varying timescales, remains to be seen.
Yeah, in reality it seems that DeepMind are more the good guys, at least in comparison to the others.
You can argue about whether the pursuit of "AGI" (however you care to define it) is a positive for society, or even whether LLMs are, but the AI companies are all pursuing this, so that doesn't set them apart.
What makes DeepMind different is that they are at least also trying to use AI/ML for things like AlphaFold that are a positive, and Hassabis' appears genuinely passionate about the use of AI/ML to accelerate scientific research.
It seems that some of the other AI companies are now belatedly trying to at least appear to be interested in scientific research, but whether this is just PR posturing or something they will dedicate substantial resources to, and be successful at, remains to be seen. It's hard to see OpenAI, planning to release SexChatGPT, as being sincerely committed to anything other than making themselves a huge pile of money.
Hao is not just a "ai is bad" book... Those exist but Hao is a highly credited journalist.
I’m not sure you understand what AGI is given the citations you’ve provided.
Isn't the point that DeepMind is producing products providing value to humanity, where AGI looks like something that will produce mainly harm?
> "While AlphaEvolve is currently being applied across math and computing, its *general* nature means it can be applied to any problem whose solution can be described as an algorithm, and automatically verified. We believe AlphaEvolve could be transformative across many more areas such as material science, drug discovery, sustainability and wider technological and business applications."
Is that not general enough for you? or not intelligent?
Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
> Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
AGI means it can replace basically all human white collar work, alpha evolve can't do that while average humans can. White collar work is mostly done by average humans after all, if average humans can learn that then so should an AGI.
An easier test is that the AGI must be able to beat most computer games without being trained on those games, average humans can beat most computer games without anyone telling them how to do it, they play and learn until they beat it 40 hours later.
AGI was always defined as an AI that could do what typical humans can do, like learn a new domain to become a professional or play and beat most video games etc. If the AI can't study to become a professional then its not as smart or general as an average human, so unless it can replace most professionals its not an AGI because you can train a human of average intelligence to become a professional in most domains.
AlphaEvolve demonstrates that Google can build a system which can be trained to do very challenging intelligent tasks (e.g. research-level math).
Isn't it just an optimization problem from this point? E.g. now training take a lot of hardware and time. If they make it so efficient that training can happen in matter of minutes and cost only few dollars, won't it satisfy your criterion?
I'm not saying AlphaEvolve is "AGI", but it looks odd to deny it's a step towards AGI.
I think most people would agree that AlphaEvolve is not AGI, but any AGI system must be a bit like AlphaEvolve, in the sense that it must be able to iteratively interact with an external system towards some sort of goal stated both abstractly and using some metrics.
I like to think that the fundamental difference between AlphaEvolve and your typical genetic / optimization algorithms is the ability to work with the context of its goal in an abstract manner instead of just the derivatives of the cost function against the inputs, thus being able to tackle problems with mind-boggling dimensionality.
The "context window" seems to be a fundamental blocker preventing LLMs from replacing a white collar worker without some fundamental break through to solve it.
It's to early to declare something "fundamental blocker" while there's so much ongoing research.
Elon thinking Demis is the evil supervillain is hilariously backward and a mirror image of the reality.
That one struck me as... weird people on both ends. But this is Musk, who is deep into the Roko's Basilisk idea [0] (in fact, supposedly he and Grimes bonded over that) where AGI is inevitable, AGI will dominate like the Matrix and Skynet, and anyone that didn't work hard to make AGI a reality will be yote in the Torment Nexus.
That is, if you don't build the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus, someone else will and you'll be punished for not building it.
[0] https://en.wikipedia.org/wiki/Roko%27s_basilisk
It's never been explained to me why a god like AI would care one way or another whether people tried to bring it into being or not. I mean the AGI exists now, hurting the people that didn't work hard enough to bring it into existence won't benefit the AGI in any way.
...or, depending on your particular version of Roko's Basilisk (in particular, versions that assume AGI will not be achieved in "your" lifetime), it will punish not you, yourself, but a myriad of simulations of you.
Won't someone think of the poor simulations??
"From my point of view the Jedi are evil!" comes to mind.
Why not both.
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
> LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable.
It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
They are not “correct” most of the time in my experience. I assumed the author left that “proof” out because it was obvious.
do you find a 40-60% failure rate fits your definition of correctness? I don't think they really needed to spell this failure out...
https://www.salesforce.com/blog/why-generic-llm-agents-fall-...
> I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
I wonder if Tom Phillips isn't very familiar with the history of Silicon Valley, because this is the most unremarkable thing. 60 years ago personal computers were sci-fi fantasy, and 50 years ago they were a mainstream view in Silicon Valley. 50 years ago usable GUIs were sci-fi fantasy, and 40 years ago they were a mainstream view in Silicon Valley. 40 years ago a global computer network used by almost everyone was sci-fi fantasy, and 30 years ago it was a mainstream view in Silicon Valley. 30 years ago always-internet-connected wearable computing devices were sci-fi fantasy, and 20 years ago they were a mainstream view in Silicon Valley. 20 years ago renewable energy cheaper than coal was a sci-fi fantasy, and 10 years ago it was a mainstream view in Silicon Valley.
There were a lot of other sci-fi fantasies that didn't pan out, of course, or at least haven't panned out yet, so this isn't a sign that AGI will definitely happen. Xanadu, the Information Superhighway, graphical programming, virtual reality, freedom, world peace, energy too cheap to meter, the full automation of production, flying cars, and so on. But the entire point of Silicon Valley is that it's a machine for turning sci-fi fantasies into everyday life.
After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity. Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
Can you elaborate on that last sentence a bit? How has engineering lost touch?
> Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value
I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
> I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
This is the new line now that LLMs are being commoditized, but in the post-Slate Star Codex AI/Tech Accelerationist era of like '20-'23 the Pascal's wager argument was very much a thing. In my experience it's kind of the "true believer" argument, whereas the ROI/productivity thing is the "I'm in it for the bag" argument.
Right. Nobody makes a Pascal's wager-style argument in _favor_ of investing in AGI. People have sometimes made one against building AGI, on existential risk grounds. The OP author is about as confused on this as the water usage point... But the appetite for arguments against AI (which has legitimate motivations!) is so high that people are willing to drop any critical thinking.
"The argument" ignores the opportunity cost of the other potential uses of the invested resources.
I think he got it backwards. Whisper, incredible things chatbots can do with machine translation and controlled text generation, unbelievably useful code-generation capabilities (if enjoyed responsibly), new heights in general and scientific question answering, etc. AI as a set of tools is just great already, and users have access to it at a very low cost because these people passionately believe in weirdly improbable scenarious and their belief is infectious enough for some other people to give them enough money for capex and for yet other people to work 996 if not worse to push their models forward.
To put it another way, there were many talented people and lots of compute already before the AI craze really took off in early 2020s, and tell me, what magical things were they doing instead?
> As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment).
I agree with the first two points, but as others have commented the environmental claim here is just not compelling. Starting up your computer is technically creating environmental waste. By his metrics solving technical problems ethically is impossible.
Perfect harmlessness is impossible. Thus, we cannot differentiate between harms, nor should we try. This a stupid thing to profess and I do not believe you would defend it if pressed.
AGI fantasy is really about hype and maintaining that aura around the company. It’s to get clout and have people listen. This is what makes company’s valuation shoot up to a trillion dollars.
> But AGI arguments based on EV are nonsensical because the values and probabilities are made up and unfalsifiable.
Hmm, so most businesses behave nonsensically, because they estimate future outcomes…
I’m not disputing the conclusion, but this crucial argument doesn’t seem very strong.
I think there’s a jealousy angle to Musk’s need to characterise Hassabis as evil. The guy is actually legitimately smart, and clearly has an endgame (esp medicine and pharmaceuticals) and Musk is just role playing.
I would love to have witnessed them meeting in person, as I assume must have happened at some point when DM was opened to being purchased. I bet Musk made an absolute fool of himself
Actual engineering is happening. Every great innovation is fantasy until it's real. What would you rather the money be spent on and why should anyone care? You can call any paid work exploitation if you want to.
The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued.
Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI.
And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works.
Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this.
I think we still lack decades of basic research before we can hope to build an AGI.
Admitting you need to do basic research is admitting you're not actually <5 years from total world domination (so give us money now).
Why should they care as long as selling shares of a company selling access to a chatbot is the most profitable move?
Many of the people in control of the capital are gamblers rather than researchers.
>AGI fantasy...
>I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley.
Human like thinking by machines, which I think is what most people think of as AGI was not until recently a sci-fi fantasy.
It was dealt with by Turning of Turing test fame and the main founder of computer science, around 1950 and the idea of singularity in the tech sense came from John von Neumann who was fundamental to the John von Neumann architecture used as the basis of much computing. If you assume the brain is a biological computer and electronic computers get better in a Moore's law like way then a crossover is kind of inevitable.
Dismissing it as sci-fi fantasy seems a bit like dismissing a round as opposed to flat earth ideas in a similar way.
Which doesn't mean that LLMs are the answer and we should stick all the money into them. That's a different thing.
But it's a boon to gathering investment capital and talent.
Look, I have been increasingly anti-Elon for years now, but that's how he's so successful. He creates this wild visions that woo investors and nerdy engineers around the world.
That's the whole point. If his pitch was "we can create better chat bots" no one would care.
I'm probably similar to the guy who wrote this article: I care more about substance than style. That doesn't get you ahead in this world, though. Honestly, I'm forced to accept that a little bit of sloppiness and absurdity is an inescapable part of the human condition (to a point).
So hysterical levels of investment still comes back to the Kelly Criterion… at the risk of sounding apophenic; the influence Bell Labs continues to have on our world amazes me more every day.
AI is also a blocker to art. Going to art school seems kind of pointless these days.
AI is a blocker to the human experience.
"Reading Empire of AI by Karen Hao, I was struck by how people associated with OpenAI believe in AGI. They really do think someone, perhaps them, will build AGI, and that it will lead to either the flourishing or destruction of humanity."
"I think it's remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley."
I think the use of the term "believe" is remarkable
According to the "AI experts" there are "believers" and "skeptics"
Science fiction is exactly that: fiction
For decades, software developers cannot stop using the word "magic", "magically", etc.
Silicon Valley is a place for believers
A place where promoters like Steve Jobs can, according to one Apple employee, distort reality^1
1. https://en.wikipedia.org/wiki/Reality_distortion_field
https://en.wikipedia.org/wiki/P._T._Barnum
Some people enjoy science fiction and fantasy. Others may not care for it. It's a matter of taste
One symptom of AGI fantasy that I particularly hate is the dismissal of applied AI companies as "wrappers" - as if they're not offering any real technical add on top of the models themselves.
This seems to be a problem specific to AI. No one casts startups that build off of blockchains as thin, nor the many companies that were enabled by cloud computing and mobile computing as recklessly endangered by competition from the maintainers of those platforms.
The reality is that applying AI to real challenges is an important and distinct problem space from just building AI models in the first place. And in my view, AI is in dire need of more investment in this space - a recent MIT study found that 95% of AI pilots at major organizations are ending in failure.
Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White!
https://en.wikipedia.org/wiki/Demis_Hassabis
> And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value).
This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI.
They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%).
You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
Yeah, but their past history should be taken into account here. Altman and musk are just confidence men. what they’ve touched in the past has turned to crap, and it’s only been the people around them that have made anything work despite those people mucking it up.
trust past history as an indicator of future action. In this case, sure some neat stuff will come out of it. But it won’t be nearly what these people say it is. They are huffing each other’s farts.
Do they believe that, or do they need to project confidence in that vision to hit investment targets? I can't tell. Probably a mix.
Money influences thinking so undoubtedly it's a mix, but I think a lot of HNers discount the former, when it plays a very large role. E.g. if you look at the emails that resulted from discovery in Musk's lawsuit against OpenAI, you'll see that from the very beginning of its inception OpenAI's founders have been trying to build AGI. This wasn't a marketing term that was made up years into OpenAI after it blew up and needed to dance in front of investors. This was an explicit goal of OpenAI from the very beginning.
That's still Pascal's Wager but with different (better) probabilities.
Pascal's Wager is based essentially on low probabilities. You can't really say something with different probabilities is Pascal's Wager.
"Probability of X > 90%. Therefore I act as if X will happen." is not Pascal's Wager. That's a bog-standard reiteration of any belief.
What is the definition of Pascal's Wager in your mind?
To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
https://www.youtube.com/watch?v=A-b7-fLOjlY
I always found it funny OpenAI staff tried to delay the release of GPT to the world because they feared the consequences of giving the public such a power. Hearing stuff like this makes it even funnier:
> In the pit, [Sutskever] had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained represented a good, aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. … Sutskever doused the effigy in lighter fluid and lit on fire.
Sutskever was one the people behind the coup of Sam Altman over AI safety concerns. He also said this in 2022:
> "It may be that today's large neural networks are slightly conscious." [1]
A good question is are these AI safety proponents a bit loony or do they actually believe this stuff. Maybe it's both.
[1] https://futurism.com/the-byte/openai-already-sentient
We don't know enough about consciousness to be able to conclusively confirm or deny that LLMs are conscious.
Claiming otherwise is overconfident stupidity. Of which there is no shortage of that in AI space.
That's the sort of convenient framing that lets you get away with hand wavy statements which the public eats up, like calling LLM development 'superintelligence'.
It's good for a conversation starter on Twitter or a pitch deck, but there is real measurable technology they are producing and it's pretty clear what it is and what it isn't.
In 2021 they were already discussing these safety ideas in a grandiose way, when all they had was basic lego building blocks (GPT 1). This isn't just a thought experiment to them.
Keeping up the ruse is the only way to justify the sunk cost in major investment.
I get the skepticism about the dramedy of burning future AGI in effigy. But given humans are always a dramady, I don’t judge odd or hyperbolic behaviors too harshly from a distance.
It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with.
I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like?
AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier.
This isn’t writing. Although that changed everything.
This isn’t the printing press. Although that changed everything.
This isn’t the telegraph. Although that changed everything.
This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything.
This is intelligence. The meta technology of all technology.
And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere.
The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress.
It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way.
Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”.
Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve.
It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think.
Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity.
Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding.
Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like.
Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us.
What an odd and transparently motivated thought.
It is such a pure thing when an engineer looks at the world and is surprised, frustrated, or disappointed at behavior at scale. This is a finance game which in itself is a storytelling / belief based system. It might seem like math, but when you're playing on the growth edges valuation is really is about the story you tell and the character of the players. Thats only worse when people stop caring about cashflows or only expect them to happen "in the future" because that makes it someone else's problem.
Where is all the moral outrage that completely stonewalled technologies like human cloning? For what most businesses want out of AGI, it's tantamount to having digital slaves.
Likely because (very?) few would associate LLMs in their current form with "digital slaves". Attributing personhood to a non-corporeal entity is likely a multi-generational change, if it ever happens.
While I agree that the current LLM-based approaches won't get us to (sentient) AGI, I think this article is missing a key point: the entire modern AI revolution (while founded on research work esp coming from Google) was fired up by the AGI dreamers at OpenAI with GPT3+ then ChatGPT etc. They were first in industry; they created the field.
Even if you don't expect them to get us over the final line, you should give them credit for that.
The lay misconception and wrongly attributed revolutions, discoveries, inventions is so common it has a name - Stigler’s law of eponymy.
You confound the AI product with the AI revolution.
You confound my point with some other of your own misreading.
To repeat: The claim of the article is clearly incorrect. It's precisely the AGI "fantasists" that have engineered the incredible technological landscape we now live in.
If you disagree, it would be useful to frame your argument in a clearer and less condescending fashion.
Thought this was going to be a good article then the author started mentioning water consumption and I stopped reading.
AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent.
In the meantime, 100% agree, it's complete fantastical nonsense.
Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing.
Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..."
What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless.So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
sorry to reply again, but it also sounds as if the lack of context is causing a problem. The word weird terms on a certain emotion and tone of voice. If this were in person, the other party might have a tone and demeanor that shows that word "weird" indicates a trailing off, a need for pause and contemplation, not a potential pejorative.
questioning someone in an academic matter further, just revert to the academic literature around psychology and therapy, where someone reflects in a literal way upon what they said. The LLM could easily have responded that it was just a trailing stray comment meant to indicate inquisitiveness rather than deflection. if this were real intelligence, it might take a moment to automatically reflect on why it used the word “weird“ and then let the user know that this might be a point of interest to look into?
it sounds like they are trained to be a confidence man executive. hype things and blow smoke. It's able to form a response when questioned carefully about the patterns created; that’s the only plus I am seeing from your point of view on this particular use of the technology.
It's intellectual charlatanism or incompetence.
In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks.
In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence.
Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater.
Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
uh, yeah no shit
>…Musk would regularly characterise Hassabis as a supervillain who needed to be stopped. Musk would make unequivocally clear that OpenAI was the good to DeepMind’s evil. … “He literally made a video game where an evil genius tries to create AI to take over the world,” Musk shouted [at an OpenAI off-site], referring to Hassabis’s 2004 title Evil Genius, “and fucking people don’t see it. Fucking people don’t see it! And Larry [Page]? Larry thinks he controls Demis but he’s too busy fucking windsurfing to realize that Demis is gathering the power.”
There are some deeply mentally ill people out there, and given enough influence, their delusions seem to spread like a virus, infecting others and becoming a true mass delusion. Musk is not well, as he has repeatedly shown us. It amazes me that so many other people seem to be susceptible to the delusion, though.
Baker act these people.
Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break.
People have been shitting on AGI since the term was invented by Ben Goertzel.
Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places.
Yet we’re making regular progress towards it no matter what you want to think or believe
The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy:
- Gödel-style incompleteness and the “stability paradox”
- Wolfram's principle - Principle of Computational Equivalence (PCE)
One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence.
The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
The human brain existing is proof that "Gödel-style incompleteness" and "Wolfram's principle" are not barriers to AGI.
"As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)."
As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics.
On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
Okay, so come up with an alternative, it's math, you can also write algorithms.
I can’t test them, though.