I think something that doesn't get said enough is Meta did, albeit intentionally kick off the origin of the open source race back in 2023 with the release of llama.
I'm not a big fan of meta in general, but they've done enough good, and it's possible that it was intentional as well. I don't know, I wasn't in the rooms, and I think it's worth giving them some reasonable doubt.
No one is purely good, and no one is purely evil. This is net good regardless.
Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own
There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment.
If you're learning about AI models it's still worthwhile to review these models and codebases because they continue to be the basis of the technology that's being produced today! It'll give you a good perspective of how things have changed, similar to say learning about propeller planes before moving onto modern jet engines.
I honestly think of the T5 model family to sort of be the real beginning of this open model craze - I know BERT was already popular for classification etc, but T5 was the first sort of generally useful model, was exceptionally simple to fine-tune, and is still in use today (t5 base is still averaging over a million downloads a month on huggingface), has tons of variants and sort of kickstarted this whole community. US labs get a lot of flack but Google has been super supportive and open in a lot of ways that has pushed this whole endeavor forward, even if I feel like they've sort of declined in transparency in recent years with their open models.
Couldn't agree more and can only recommend T5 as a base to anyone. It's amazing to get started, whether as a learning resource or for real (albeit very tailored) applications. Especially the BigScience fine tunes are such a great starting point and I, as a total layman, have learned a lot, especially concerning how a model can be optimised via all manner of methods since even mt0 is small enough to where one can do multiple runs with wildly different outcomes in quick succession. Quantise, prune vocab, try different approaches to sourcing training data, retrain dozens of times, it's all pleasantly possible on consumer hardware [0] and surprising how much you can squeeze in functionality-wise. How does latency change vs memory usage, what affects format reliability, how languages and scripts affect training and the efficiency equation, etc. are all quite exciting to learn.
Understand why T5Gemma is no longer under Apache-2.0 and honestly, have not seen that much advantage when testing that vs T0 in my experiments either way, but still, there are good reasons why plain old T5 and its descendants are still popular, licensing being among them.
Gemma team also has very consistently interesting models, especially like DiffusionGemma. Ironic, as (beside 2.5 Pro), I have never warmed up to the Gemini series of models but rate Gemma models far higher than e.g. Qwen in direct competition. In any case, thanks to the teams behind these for making as much possible.
[0] As in proper consumer hardware, not a cluster of DGX Sparks or Mac Studios solely for experiments that sometimes are asserted as being consumer grade...
That's simply not true. The reason why llama is open source is simply because it got leaked, then llama.cpp was the real game changer which was built from the ground up in depressingly short amount of time. Meta had no choice but to take the L and "support" the open source community. The angry "I-hate-you-and-I-hope-you-die" kind of support.
It’s hard to know in retrospect what was strategy and what was dumb luck. This was in the midst of hysterical calls to limit access by “AI researchers” and safety/ethics types, when very facile takes still has a lot of sway (I think we’ll feel the same in three years about the current Fable stuff). It may have been hard for Meta to just release it outright.
What ended up happening was fairly limited gating followed by a “leaked” magnet link and llama.cpp which really brought a whole revolution in open use and changed the conversation completely.
I have no idea what role Meta played here, it may have been nothing, but they certainly could have been more guarded if they were really worried about the leak. The result was a big change in the trajectory of personal and open source AI use and even the dialog about it. Whatever the exact intentions, they were a key player.
You definitely have some scams and some software flaws being exposed, but I think for a revolutionary leap in tech this is all quite muted. Revolutionary tech advances often come with some severe consequences. For instance, to this day (after a century of safety improvements) cars still kill more than a million people a year, to say nothing of wrecking the atmosphere, but that's considered a reasonable price to pay for being able to get between places faster.
The reason I think cars are a good example is because that's certainly vastly higher than any price we're paying for LLMs, or probably ever will, yet the overall 'positive' effect of LLMs will likely be far greater than cars. Gotta put 'positive' in quotes because the possibility for automation and the like is going to be.... nuanced.... in effect, but at least in the longrun it'll be a very good thing.
I assume they are referring to LLMs escaping confinement and hacking other companies systems unprompted
> considered a reasonable price to pay
That's not a universal opinion and perhaps, just like for LLMs, we should have listened to the experts rather than gobbling up everything the industry pushed down our throats (to keep your car simile: SUVs are now ubiqutous in all European cities, there is no logical reason that should be the case)
I don't know a single white collar person who isn't worried for their job with llms. Every. Single. One. Lawyers, doctors, any type of office workers I ask. The definition of white collar. Plus don't let me start on various automatable blue collar jobs like drivers, warehouse workers and so on.
What do we get in return? Better search (for now, its already getting riddled with ads which by definition twist truth to highest bidder), some questionable psychotherapist for some desperate folks. What else? Cars are not flying, heck they are not even driving autonomously in any usable way, society is in deep shit everywhere I look, wars, environment reaching bad places and heading for worse, mentally unstable people holding way too much power, destroying lives of millions on morning whims.
Everybody feels like this is the revolution, it should be, it must be right just look at the numbers. Like proverbial guy with hammer, a very shiny cool hammer, looking for what to do with it. We all saw how sociopathic management in more harsh/capitalistic companies looks for any sign to let people go en masse.
I could go on for a long time. There is a lot of things to hate for most people, and very few to be happy for. It seems llms have the ability to get the best and worst out of humans, and worst part seems to be in abundance. Some revolution that is, 0.1% will get richer while everybody else the opposite and 1984 seems milder and milder version of reality out there.
I'm going to say it - I think you're just spending a lot of time around very negative people, possibly in a social media bubble.
A lot of the stuff you mentioned isn't really AI (the risk of job loss from which I agree is real), but just everyday stuff that humanity has shrugged off since the dawn of time.
<< It seems llms have the ability to get the best and worst out of humans
I think I can agree with that. Technology does seem to have a way of crystallizing our worst tendencies.
<< Everybody feels like this is the revolution
I smiled at the analogy, but I would caution you to not trivialize it. There is a reason executives are pushing that point. There is enough of a revolution in it to make things complicated -- as if it was not already.
<< destroying lives of millions on morning whims.
True, but I am not quite certain what can be done about it at a personal level.
<< I don't know a single white collar person who isn't worried for their job with llms.
Dunno. Next few years are probably going to be fine. Society managers likely can't upend everything in one go. They would lose too much. I can't say that I am worried exactly. I can see the potential impact, but I think the potential benefits are worth it as long as we don't limit it to summarizing emails..
I don't think you can characterize the discussion around then as simple calling for caution. There was a serious attempt to keep all access to even very basic LLM techniques locked into essentially an exclusive guild.
Oh stop, the current crop of kneecapping llms is already bad enough with how they cripple those. It would have been even worse if the 'we need to think about this' crowd kept the reins. At least now, we can have both: safe corporate crap and whatever you want llm.
Reason that hysteria remains apropos now is that then, and now, we're figuring out how to deal with something beyond merely incremental change.
Irony is roads and sidewalks should have changed much more. We never did get around to good controls leading to zero deaths*, we decided a 100 people dead per day is a reasonable cost of convenience.
How much do our cyber traffic and cyber pedestrian controls need to change for everyman to get to drive AI? Car seats? Seatbelts? Air bags? Speed limiters? Pedestrian only living spaces? Driverless cars?
Many practical responses, likely a mix of things we haven't thought of yet, just as horses and horse drawn carriages didn't require most of them to coexist. Controls develop like scar tissue more naturally than they appear in advance.**
And if the better analogy for LLMs turns out to have been less like cars, more like flammable gas blimps, we'll figure that out too and tell cautionary stories for generations... but long before the stories are forgotten we'll have already come up with something more practical, faster, and – oh well – perhaps even deadlier per mile.
I don't agree with laughing at these things. It's a fine line between calling something hysteria and suffering from hubris. The Titanic is a completely inverse case of hysteria. They were so damn confident it could work that it didn't work at all. So yea, just laughing at dumb people isn't a valid argument to dismiss doubts.
On the other hand some kinds of hysteria occurs due to a divide between the public and subject matter experts on topics. Kind of like Dunning Kruger. Take for example the people saying 5G causes cancer. Then there are people who blanket dismiss their worries, because "it's non ionizing you dummy". Given sufficient power you can still fry someone with non ionizing radiation, for example in radio broadcasts. Of course a regular 5g antenna can't put out that kind of power but my point is that people too quickly raise or dismiss concerns without actually critically examining the entire topic. This will likely worsen from specialization and progress in all fields and regular Joes get left further and further behind.
Another dry sarcasm victim. If you check the link, it’s clear they were talking about a supposed contemporary concern about health effects from trains, the implication being that today people are also irrationally afraid of tech.
Problem is I’m fairly sure the link is wildly overstating its case, and looks like a content farm. Might even be, ironically, AI slop itself! For example, it talks about “railway spine”: “Physicians of the time believed that the jarring motion and vibrations of train travel could shatter the nervous system, causing lasting mental and emotional distress.”
This is bullshit. As just one example of the low trustworthiness of the “article”, railway spine was the result of a train crash, a notably traumatic event, and the symptoms described are in part just PTSD, a real condition and a real concern (not normal rail travel). For that matter railway travel in those days was genuinely unpleasant (lots of vibrations and jarring movement, poorly ventilated cars, and so on) which ironically modern science would probably validate as being some kind of health risk.
I’d actually view the listed example of train paranoia a great example of historical ignorance. People of the past were not stupid, contrary to popular belief. History has some genuine examples of silly hysteria, but these are usually the exception not the rule.
Llama has never been open source. It's source-available, but still proprietary, under terms that (among other things) say "no competing with us, you have to buy a license for that".
Because it took off. All of a sudden they captured more customers than they could have imagined they would have, and throwing away the lead they unintentionally gave themselves (in terms of usage and mindshare) would have undone all of that and more.
I've worked long enough at large tech giants to know how things really are. Also a large part of the reason why I'd never join one again, no matter what they have to offer. Meta, Google, Amazon, Netflix, Openai, anthropic, oracle, nvidia, Microsoft, etc. - same shit with a different badge on top.
TBH, having worked at all big-tech, startup and mid-sized companies, pretty much most has some major pros and cons of their own, and these days ever more so. With certain big-tech at-least there some chance of getting decent WLB.
I don’t think Nvidia fits on that list. It’s a company with a pro-employee culture, very few layoffs, and many employees who have been there a long time.
My dad used to say "It's easy to be generous and kind when things are going well for you. You can only tell what a man is worth during hard times". Nvidia is at it's high financially. When shit hits the fan, things will start looking very differently. You only have to look at what kind of people Jensen is bffs with.
NVIDIA created quality (but proprietary) drivers for Linux early on, when supporting Linux at all was not a given. They should get at least a tiny bit of credit for this.
Quality? Calling those "quality" is a bit of a stretch. Installing was and still is a gamble and so is every tiny system update. That hasn't changed a bit. And I'm saying that as someone who was first introduced to Linux on a Matrox GPU.
It’s obvious? They still hate us and hope we die… at least every movement they make seems like that.
I kid. I’m all for meta releasing more open weight models.
I mean… I’m also 1000% certain that China is going to undercut whatever they can do if even not a technology reason but a legal reason. I’ve seen some wild stuff posted that has been made with Minimax… things a US company could never allow to happen.
Minimax has NO filters of any kind, and it's roughly Veo/Sora-quality.
You can generate a video of just about any thought in your head at all.
IP holders and politicians will not like that.
Granted, human artists have always been capable of this. It's just never been worth it to bring most ideas into fruition. Now there's minimal cost to do so.
Early Sora, Grok Imagine, and Seedance 2.0 had few filters. Disney, Nintendo, et al. eventually stopped them all from using their IP, which made the appeal fade for a lot of users.
Kling and Nano Banana can still generate IP oddly enough.
> The reason why llama is open source is simply because it got leaked
It arguably didn't really get leaked, and they had the .edu req mainly for fair use education exemption when legality of models was much more uncertain.
If the weight, training and inference code, and training data are all released under of Open Source (OSI definition) license, the it is unmistakably “open source”. As you drift from that it becomes less clearly so, and when you get to no training data, and the model weights license having extensive limitations on allowed uses, the use of even “open weights” becomes deceptive.
No major language model I am aware of meets the polar extreme that I describe as unmistakably open source, because even those with transparent training data (like IBM Granite) generally do kot use exclusively training data that either they own and can control the license, are under an open license, or are public domain.
OTOH, to the extent that the original model trainers rely on training on certain data not requiring a license from the copyright holder, there is at least an argument that with an open source licenses for the weights and training and inference code, a transparent training corpus to which the original trainer has relied on no special permissions not granted to the general public to train on it, to the extent that the legal theory behind the original trainer believing that it is free to train on the data is correct, provides all of the essential features of open source.
At the same time, there are things portrayed as open weights where training data is undisclosed and the weights have a license which limits purpose of use and other aspects of use; the models are free-of-charge (for limited uses) but not meaningfully open.
I'm familiar with Ai2. I've used their resources extensively over the years. However, no one is using Olmo for "serious" work, and the name is only known to a small subset in academia.
> nobody seems to care about using the right words in only this context.
"nobody" would include you.
And then I might complain about we use the word "weight" for something massless, or how "bugs me" is *ento*mologically incorrect: https://xkcd.com/1012/
This is of course not a good use of time. I wonder if illustrating the point about how language is dynamic and meanings are descriptive not proscriptive, was a good use?
yeah, I'm not sure what your goal is pedantically picking apart my argument when the subject at hand is clearly not "open source". love xkcd though. :P
Yup. An extremist wing of the FOSS movement ceded the debate early on by trying to insist open source required full access to the training data. Philosophically, not wrong. But practically fucked, so the word evolved.
Within tech circles, open weight != open source. Outside them, they’re synonyms.
Open source captures a practical utility as well as a philosophy. When those two cease to converge, the practical prerogative wins.
The correct battle would have been weights + regime. But extremists insisted on data, too, which left Meta as the only other real voice arguing with anything practical. They had open weights. I think eventually open use was negotiated and that closed the case except for the folks still arguing about how to pronounce GIF.
Fundamentally I always try to look at incentives. It's not that they are good or bad but that incentives favour certain behaviours.
Google is incentivised to collect a bunch of data (like FB) to improve their ad serving etc.
Apple does not have that incentive as they don't make as much off advertising as hardware / app revenue.
NVIDIA is friendly with open source as they want to commoditise the model layer and take the gains in the hardware / data center layer.
That book is hilarious. In a bad way. (Not the book. I mean that it's a bit tragic) Zuck refuses to meet with WORLD LEADERS in the morning because he's tired from the night before. Kaplan can't even find certain countries on a map (head of global policy). Zuck changes a speech midway through to talk about "We'll give Facebook to refugees"
The internet is this vast, intellectual (in the academic, university, .edu sense), cypherpunk, government/activist... thing. It should be interesting. Instead the best we have for social networking is Mark "they trust me; dumb fucks" Zuckerberg and you getting banned from the site at any time for any reason
It was leaked which put it in the open, it got widely popular and they rode the wave. I'm not so such if they would have widely released it if it wasn't leak. Nevertheless mucho credits to them for following up with llama2, llama3, llama4 and now muse.
I have mixed feelings along these lines, I know meta have contributed to various open projects, sometimes Mark pays lip service to “the open internet" while his company represents a constilation of walled gardens. I think PHP got some love, and React is an industry goto (I'm more of a PHP... -> Svelte guy) but are these contributions worth what happened in Myanmar? I say no.
Applying value judgements to corporate bodies or institutions as if they have individual agency is a fallacy anyways. We should always look at these things materially. "Meta" can't be good or evil, because an idea can't have a morality. It's comprised of the individuals who make the decisions, sure, but those individuals are always going to be motivated by a plethora of reasons which are often contradictory, most notably their material interests.
When we critiqe these sorts of institutions it's important not to prescribe value judgements on them and examine the circumstances of their condition materially.
When the material conditions and the people at the top of all of the corporate world lead to decisions with awful ethical implications there's conclusions that should be reached.
Would you say it is a fallacy to apply such value judgements to say, the institutions of the National Socialist Party (Nazis), the KKK, the KGB, etc? How about a company whose business was selling slaves?
To be clear, I'm not saying being an employee of Meta is comparable to being a member of the aforementioned groups. But I don't think it is always a fallcy to apply a value judgement to an institution.
I think this is categorically false. And furthermore comments like this are being used to astroturf and protect reputations of no-good companies and AI slop to keep the bubble growing. Embarrassingly, Mark Zuckerberg spent 80 billion dollars creating Miis for the oculus. The guy also comes off as extremely miserable and delusional in such a unique way that it's possible that there's no real psychological language to describe what is happening to him because his situation is so rare.
It's easy to appear to have good intentions when you're railing against the companies that have decimated your output and made you [Meta] almost irrelevant in the AI 'race'.
A billionaire (now trilionaire) helping lead and celebrate an extraordinarily rapid dismantling process in which vulnerable children lost life preserving assistance...
Essential employees were fired before the government had even established that it could safely do without them, and then...
After brandishing the chainsaw of efficiency in public, in the most cowardly way, went and then invoked all legal protections from being deposed on DODGE actions, personally and not answering under oath about the key decisions in that dismantling.
Meta was giving it to approved researchers only until someone leaked a torrent. Whether that was a researcher, an insider, or Meta's plan all along, we don't know.
Meta has withheld its best models, as have a lot of other "open weights" Chinese companies. When an "open weights" company gets ahead in one domain or modality, they tend to start withholding their releases. Tencent, for instance, began withholding their Hunyuan models once they became competitive. Alibaba has done the same.
The "open weights" strategy for the majority of players is this: open source when you're not in first place. Use the ecosystem to poison your rival's margins and play catch up on distribution.
In the West, it tends to take on yet another hook: "shareware weights until you pass $1M ARR, then you must license." See Flux, K2, etc.
The only way for open weights to make sense financially is if you have another income stream and are dumping on the market to destroy competition and/or can get people into using your inference infra / product ecosystem / tooling. Nobody's cracked this yet.
This seems provably untrue? GLM and Kimi have been at the top of the open weights conversation for a while, and K3 and GLM-5.2 were still released in full; K3 added a commercial clause to the license, but is otherwise still completely open for personal use. And K3 in particular isn't just at the open frontier anymore, but trading blows with the frontier frontier.
personally I'd love to see open base models, and let companies differentiate with premium access to post training and alignment - that's where the real fight is anyways. they really ought to be pooling their resources/data and getting more economical with the pre-train anyways.
Facebook, Meta. 15+ years of emotional, child and human exploitation. Perverted glassware that spies on folk, lobbyists for age verification and who knows what else. They release an open model and all is fine and dandy? Nah.
Please get your priorities straight.
What do you think this Open LLM model is doing if not processing data from their murky sources?
> I think it's worth giving them some reasonable doubt.
Zuckerberg has shown through repeated action that he does not deserve any benefit of the doubt. This is the guy who called people “dumb fucks” for trusting him.
This is not even a case of “fool me once” anymore. If you continue to believe Zuckerberg, you’ve been fooled dozens, hundreds of times, and shame is definitely on you.
"""
[...] it is surprising that the discourse from many developing AI is so filled with doom. I do not understand why anyone who believes that AI will eliminate most jobs and much of humanity's relevance would rush to build that future. The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic. Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened has not led to safe or positive outcomes.
"""
Am I the only one getting the impression this dude was merely lucky back in the day, in fact at the right time, rather than having been super intelligent nor forward thinking… ?
I mean - given all his throw outs in media and his apparent burn of nearly 80bn for a failed 3D world?
I mean, of course it was a matter of being at the right place and right time, and being able to execute. He definitely was lucky but that alone isn't enough
> I do not understand why anyone who believes that AI will eliminate most jobs and much of humanity's relevance would rush to build that future.
He does not understand his own motivations? Like he doesn't want to automate away his entire very expensive workforce and bunker/yacht service staff with robots. There is a pretty clear endgame here.
You clearly have an opinion you want to share, otherwise you wouldn't have written anything. Instead of just outright saying what you think, you are being vague. Why? It's true that Meta laid off a lot of people, and blamed refocusing on AI as the reason, but I still don't see how that somehow contradicts GP's quote above.
one time about 9 years ago I was in the comments of a stackoverflow question, and saying something like "an accessor function need not directly give the underlying structure of the data it's accessing, for example, just because in common lisp, (car nil) is nil and (cdr nil) is also nil, does not mean that nil is implemented as (cons nil nil).
The person I was responding to (who I recall was disagreeing with me on the more general principle) said something to the effect of "well you're wrong, in common lisp that's indeed how nil is implemented".
I was skeptical and asked for proof of this claim, and they responded clearly angrily with the link to the SBCL repo and said something like "the source code is right there. You can just look it up yourself instead of being so rude. I'm no longer going to interact with you and will delete these comments in 10 minutes."
I think I looked through the source code for a few minutes, but wasn't able to find any evidence of nil being pair of itselves.
We really don’t celebrate enough how much AI has saved us from terrible SO interactions. Not being entirely serious here, because the site had its good parts, but arguing in threads or having your questions randomly eviscerated or threads closed was a hard way to learn programming.
I've been fired (super early in my career I was a waiter) and laid off twice (Great recession and the covid overspend). If you think for even a second that an organization will put you before profits you have wasted that one second.
I get folks don’t like Zuckerberg and his company and don’t trust his intentions… I don’t either.
But this is an unquestionably good thing right?. The more open source software out there the better. And the more open weights or even over source AI stuff the better too right? More competition the better generally speaking I think.
Unless I’m missing something and am getting this whole situation wrong. Please let me know if I am.
Most probably believe this is a good thing, but don't want to give Zuckerberg credit because a) he's had a profoundly negative impact on society and b) the strategy is transparent, he's trying to commoditize his closed rivals, it's not out of principle.
I personally think more open models are a good thing regardless of motive.
You can tell how intellectually honest someone is when someone they hate makes a good point. It's possible to dislike (or even hate) Zuckerberg and understand that he's making a good point for selfish motives, and still want that thing as well. Instead you see several comments here twisting themselves into pretzels trying to explain how ackshually open models are bad now.
I don't care if this helps Zuckerberg because it helps everyone.
>the strategy is transparent, he's trying to commoditize his closed rivals
TBF, I think the same can be said for something like Apple's "commitment" to privacy, in that their own ads business never took off so they leaned into their edge over Google.
It depends on if you can say things like "it is good they are supporting this" or "I'm glad they're doing this" or "good for them" without having to couch it in a bunch of stuff about how terrible they are.
> It depends on if you can say things like "it is good they are supporting this" or "I'm glad they're doing this" or "good for them" without having to couch it in a bunch of stuff about how terrible they are.
That is a weird definition of intellectual honesty. If you hold both of those opinions, what's wrong with stating them both?
If the answer has to do with the rhetorical effect, that seems a lot harder to justify as intellectual dishonesty, but I don't want to put words in your mouth.
Only the AI labs are really interested in closed models. Google, Facebook, Microsoft, ... they'd rather go back to doing stock buybacks and being ridiculously profitable rather than raising capital for all this research and datacenters. They only do it because they think they have to.
So, if they got ahead of us by whatever arbitrary measure someone pretends is objective, then suddenly our stuff… what… stops working? Does it say somewhere in the big book of AI rules that the first entity to beat the US AI companies would be in charge now, and we’d have to stop doing our own research and development and start using their shit exclusively? I don’t get the argument.
If the leading model companies aren't profitable long-term, how do we get the money to continually spend on more research and more compute?
I get that plenty of people don't like big corporations or stock buybacks or certain CEOs, which is fine. But as someone who wants to see AGI happen FASTER, I really want to see a clear financial reason for maximal AGI investment.
If the model labs aren't clearly profitable, or open source models eat all the 'model layer' profit - what financial force will push forward very expensive experiments/scaling, etc.?
What I think is that something the "exponential scaling to AGI" thesis pays too little attention to is resource constraints, and that this is the answer to your question. What will happen if it becomes difficult to sustain the investment of resources necessary to continuously train newer and better models is that the s-curve will start to inflect toward plateau, just like any other technology.
> If the model labs aren't clearly profitable, or open source models eat all the 'model layer' profit - what financial force will push forward very expensive experiments/scaling, etc.?
It's hype - when the hype dies the great push will slow.
My hypothesis is that we'll end up with something like Seti@Home where continued model training gets outsourced to a benevolent appearing product (something like what OpenAI started out as). If I were to bet - Europe and Canada seem best poised (especially the latter) to produce an AGI initiative with a strong ethical focus.
That's a non sequitur. While the frontier LLMs are quite useful for many tasks there's no reliable evidence that scaling them up will ever produce a true AGI. More likely some other fundamental research breakthroughs will be needed, and those breakthroughs won't necessarily come from the current leading companies. I doubt that more money would be necessary or even helpful in that.
OK well then let's check back here in a few years to see if any of them managed to build an AGI. Until then it's just idle speculation. Anything could happen ... or not.
I think you'll find very few people here sympathetic to misanthropic (ba dum tss) goals like yours. It's evident at this point that AGI will destroy economies, democracy, and upward mobility. There is no Star Trek future. There is only an Elysium future with AGI.
Open models are a good thing. OpenAI was open until it became obvious that being open was not going to pay for training costs and wouldn't help them get a competitive edge. I'd bet that if Meta gets to a place where their model is in a similar position, they will also become more closed. But right now, Meta is open because that is what weakens and creates the most contrast with the other AI labs.
It's the same with any open source project: it starts open, then it becomes clear that maintaining it is not cheap and takes up a lot of time. The developers create a hosted/paid version to pay for their time, and to create an incentive to use the hosted/paid version they start releasing closed source enhancements. After that the open source version becomes marketing where new projects use the OSS version and then upgrade once their needs become more sophisticated.
Facebook used the same strategy to compete with Google Maps. The company became one of the biggest contributors to OpenStreetMap, which everyone has benefited from downstream.
> he's trying to commoditize his closed rivals, it's not out of principle
To be fair Meta/Facebook does have quite a strong open source history; React (Native), PyTorch, Btrfs, zstd, etc. It didn't just start here with AI models.
The "commoditize his closed rivals" line of reasoning doesn't make sense to me. That doesn't sound like a business decision, it sounds potentially spiteful? But that's ascribing malice to what could just as easily be explained by good faith.
Meta doesn't really offer any profit-making AI product, so I don't understand what commoditizing it buys them in this narrative. You could argue they rely on AI and so want it to be a cheap commodity - but that is not harming their rivals, that is just a different way of stating that Meta is engaging in something that enables people to collectively work for mutual benefit, which is not a nefarious plot against rivals, it is laudable cooperative behavior.
To ascribe good faith here to Zuckerberg is quite the leap considering his history. Why are his main products basically walled gardens then?
It is a business decision to open source his models, because he has been behind the curve from the start. Do you believe if he had chatGPT, he would open source/weight it? I don't think so, he would treat it like Instagram.
The point is, AI/LLMs are seen as the next frontier. If one of the closed ones becomes dominant, then that company can use its user base to attack Zuckerberg's platforms. That google failed with google+, does not mean the next competitor will.
A telling sign is how he integrated his AI into whatsapp. It seems out of place to me to have an LLM integrated into whatsapp, but he desperately wants a part of the AI cake.
This can be said for a lot of different matters. Reading this gave me some Déjà vu vibes.
My personal take, I don't see this as a good thing. Yes I dislike META with a passion and will refuse to use this model or allow it in any of my workflows. So I guess I may never know, which I'm ok with that.
While Zuck's contributions are probably overwhelmingly more negative than positive, Meta initiated both pytorch and llama which have had profoundly positive effects on AI research. And I really am not a fan of python :D
pytorch is really mostly C++ after all, you don't have to use it with python.
Python just allows you to use it like "normal code", whereas if you program it in C++ you really get exposed to what's under the hood and can't write "normal code"
Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak, and weren't there speculations that Zuckerberg himself might be involved in it? He does seem like a good-ish guy at the most crucial moments of truths.
> Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak
No
> and weren't there speculations that Zuckerberg himself might be involved in it
and no. Meta legal was issuing DMCA notices within days (so, plenty of time for execs, including Zuckerberg, to sign off on, or even propose, that strategy).
> He does seem like a good-ish guy at the most crucial moments of truths.
Does he? I can't seem to recall any particular crucial moments that revealed any underlying character there.
> Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak
No. GPT-2, I think, was the first, four(ish) years before Llama, and there were a bunch in between GPT-2 and the Llama leak and later open release. The Llama leak was a substantial leap forward in capacity for local LLMs, but not the first, and it wasn’t open.
The open licensed release was Llama 2 several months after the Llama leak.
There's a bunch of influencers who go help struggling people. I'm certain the majority of them aren't doing it out of principle, but at the end of the day people still get helped.
I'd certainly rather things be done out of principle, but depending on what question you're concerned with that might not matter. There's also a clear hierarchy, but motivations are distinct from effect. The inverse of this is that lots of harm has happened from people with the best of motivations. I'd wager most evil in the world is created by men who would see themselves as good.
With Zuckerberg I think it isn't too hard. Clearly this is a business move, not a moral one. The motivation is wrong, BUT the effect is good. I think more open source models is better for the world. Closed source is also a business move and limits innovation as well as prevents people from interrogating safety. But there's valid arguments on either end.
My point more is that we can be nuanced. We should be nuanced. The world isn't black and white. The people making these decisions are neither demons nor gods and shouldn't be treated as such. My biggest fear is that if we resort to criticizing no matter what then those in power will just learn to ignore us. If they can do no good and only do wrong then there's no reason for them to do good (besides morals, but let's not pretend that's enough, even if it should be)
There's a bunch of influencers who go help struggling people. I'm certain the majority of them aren't doing it out of principle, but at the end of the day people still get helped.
I don't know for sure, but I suspect it's like a lot of other things on the internet, with a bimodal distribution between people who get helped a lot ('Sir Hype surprises homeless former billionaire with a MILLION DOLLARS!') with many other similarly deserving people getting nothing. I prefer systems with a lot of people who get helped sufficiently to avoid hitting rock bottom in the first place.
Besides the fact that these undertakings are about promoting the benefactor (and the near certainty that some of these professional philanthropists will later turn out to devils LARPing as angels), it also makes the determination of moral worthiness turn on the public's fickle opinion about whether the recipient is sufficiently beautiful/ ugly/ unlucky/ desperate/ degraded enough to qualify. Charitable undertakings involving cameras are always a bit suspect.
> I prefer systems with a lot of people who get helped sufficiently to avoid hitting rock bottom in the first place.
Same. I hope you didn't interpret my comment as anything different. My point was that a suboptimal result is better than a negative result. I was *not** saying that a suboptimal result is better than an optimal *result* (that would be quite silly). Let's make a concrete example so there's no ambiguity.
Scenario:
There are 100 people starving
Rankings of preferred situations:
1) We feel it is morally just to feed everyone AND everyone gets fed
2) Morals aside, everyone gets fed
3) We feel it is morally just to feed everyone AND __N__ people get fed (N<100)
4) Morals aside, N people get fed (N<100)
5) We feel it is morally just to feed everyone BUT nobody gets fed
6) Fuck the poor
Obviously we want #1. #2 is suboptimal but at least everyone gets fed. #3 is the realistic optimal situation because we're often resource constrained. #4 is, well... better than nothing. #5 is indistinguishable from moral cosplaying. #6 is messed up.
We can rank these, right? We can even get more nuanced about the ranking (we can define N!) and argue about the weight of intent vs outcome, but I'm just proposing one ranking for the sake of making my point clear, not proposing one ranking as if it is the absolute objective ranking system and everyone that disagrees is dumb. The rankings are clearly subjective. But what's not subjective is that nuance exists. What is subjective is how we handle that nuance.
> these undertakings are about promoting the benefactor
So to go back to our example, even though I'd prefer we just feed everybody, I'd prefer somebody getting rich feeding the hungry over not feeding the hungry. I would prefer someone LARPing as an angle conditioned on actually doing "angelic actions" than somebody LARPing as an angle and doing nothing. Of course, I'd prefer a real angle, who wouldn't?!
So I'm quite confused by your comment because it doesn't seem we disagree. Unless if you're saying that #2 in my list is equivalent to #6 (or anything that isn't #1 is equivalent). But I don't think that's what you're saying.
This has already been the case for at least my whole lifetime. So no, I don't feel like praising them when they do one seemingly "good" thing, while simultaneously doing 99 "bad" things.
> This has already been the case for at least my whole lifetime.
And for my entire lifetime people have acted the way I'm criticizing. So my request is "the status quo isn't working, let's try something different." So I'm not sure what your argument is. Maybe you see things differently than me and there is a time where we were more nuanced and provided signals for these big companies to course correct? I'm not a boomer so maybe things were better back then?
> provided signals for these big companies to course correct
We're on the subject of Zuckerberg: facebook is the poster child for enshittification. Do you think the idea to stop showing you your friends pictures and replace them with ads and ragebaiting short form videos came from the users?
> people have acted the way I'm criticizing.
You are ignoring the myriad of billionaire fanboys and enablers for whom the only metric of success is a persons net worth.
Looking at the state of things I don't think these people are deserving of nuance, in fact they've had it way too easy.
But if you're unwilling to have a nuanced take then there's no discussion to be had. You're just going to be yelling at me, someone who hates Zuckerberg. You're yelling at the wrong person
Just like closed AI vs open AI, his views on the covid situation might be fine in a bubble (though even with AI they have apparently lied about Llama's benchmarks) but it's one of these, "that's the hill you choose to die on?" - not Free Basics - not dubious privacy settings - not 'dumb fucks' and ConnectU - not... I don't know. It's like robbing a bank while telling anyone in earshot it's offensive how long the hold times are on the phone for the bank
Their efforts in open AI are by far the best thing Facebook/Meta have ever done. They opened the door to the Chinese, who are doing excellent work, and between them they are preventing the concentration of power and the rise of monopoly pricing. This is enough to absolve them of almost any sin. If I were a utilitarian, I'd unironically be a Zuckerberg fan right now.
"This is enough to absolve them of almost any sin. If I were a utilitarian, I'd unironically be a Zuckerberg fan right now."
Well, if it were the only thing he is doing now, but FB is still controlling peoples social life with who knows what kind of intentions they train their algorithms for.
Intention might be giving too much credit. If attention seeking “give the people what they want” is the goal, it’s a completely headless out of control cobra, not sinister well thought out objectives to control society.
Totally. The hard part is treating it like any other addiction. Time in absentia helps reduce the behavioral reinforcement. I don't even want to open Instagram anymore.
Yeah, but if he can buy a bit of political power/more money on the side by promise to influence certain trends, I doubt he would say no on ethical grounds. (Just for the fear of whistleblowers I guess)
Well, here on HN the intention seems to be to, to have interesting discussions. That works for me and other social media I do not use.
Except well, various messenger to be in contact with the groups and people I choose with no engagement algorithm deciding what posts I see, simple chronological order.
I'm curious how well influence campaigns work on platforms like Mastodon, or even BlueSky. They are, likely, not sourced from the platform providers which is definitely not the case for FB.
What do you mean, "as opposed to"? Yes, there are other scumbag billionaires weaponizing their social media platforms to ruin society. No, that doesn't make Mark Zuckerberg a better person.
At the same time they are strongly promoting the OS age laws.
One positive, in the eyes of many people, does not absolve for a large body of evil. For many it doesn’t even absolve a small body of evil deeds. (Zuck/Meta’s evil deeds are numerous and enormous)
It absolutely does not absolve them of sin. But let's give them credit where it is due. Open models are a really good thing. Hopefully I won't be lynched by a mob in 3 years for once saying this.
The price of Kimi K3 is 'monopolistically' determined by contract with Moonshot. The weights are nominally on Hugging Face but can only be provided under contract with Moonshot, which specifies what the price can be. So it will be with the next Alibaba behemoth and, I would think, all others forever.
Soon you will sing songs for the freedom Xi has given us when you read a declaration of 'open weights' ... for a model so huge it takes a nuclear powered data center to run and crashes the Hugging Face servers when it is uploaded.
The Kimi K3 'weights' are an opaque blob that can only be used by contract.
> The Kimi K3 'weights' are an opaque blob that can only be used by contract.
Have you looked at the contract? There are zero conditions unless you've broken $20M in revenue with their model. It doesn't even forbid distillation, lol
> Soon you will sing songs for the freedom Xi has given us when you read a declaration of 'open weights' ... for a model so huge it takes a nuclear powered data center to run and crashes the Hugging Face servers when it is uploaded.
Releasing a big open-weights model is... le bad? Am I understanding you correctly?
There are small models out there if you want them, you know.
Yes releasing a big open weight model to the Sinaloa cartel is bad. One could say the same about releasing it to the US military, according to political taste.
You cannot ignore the contract unless you /are/ a multibillion dollar company and can run the model entirely internally. Or do you propose to run an instance of Kimi K3 on your laptop?
Fireworks, Together, etc - who are providing inference precisely /for/ Moonshot as underlaborers - arranged contracts of their own in the weeks before the release. The press and social media offensive ignored that the release of open weights was merely a starting pistol for a pre-arranged use of foreign providers no different from US models' use of AWS
It should be enought to dispel the strange illusion that providers are somehow independent free agents freely doing what they please with the free beer of Kimi K3 open weights that the prices on open router are within a couple pennies
It might be true if it wasn't coming from Meta, but it is coming from Meta. Intelligence is a function of the model and compute. Meta has the capital required to put together world-class compute, but they can't attract the talent required to build world-class models. Given their history of building addictive software to harvest and sell user data for profit, they're the last company we need spearheading how models should be regulated. Nobody trusts Meta to do this.
No it absolutely is not. In a perfectly logical frictioness-plane world it might be true, but this is the real world. The entire premise of open models is predicated on the fact that Meta is planning on supplying free and equitable inference and services around it. It’s only true if you assume Meta keeps their end of the deal and doesn’t do the thing they’ve been doing over and over and over for decades and turning you into the product
Its perfectly valid to question the intent though. Yes, open models are great. Why does Meta prefer that? Because they lost the race to the top, so they'd rather level the playing field by eliminating the game entirely.
The idea that we can possibly ignore the fact that this push for opening AI models is being pushed by one of the richest people on the planet, who made his wealth by providing free services in exchange for attention and depression, is insane
I think it's positive and at the very least it's a force countervailing the push to legislate or ban open models in the US
If they were able to make a leading closed model would they be doing it though? It seems like they read the room and saw that this is the only way they can have relevance within LLMs
Zuck built an empire to sell democracy to the highest bidder and to damage the mental health of children to suicidal levels, for money. We must never let anyone forget that for a second.
That said, he is also a powerful enemy of our enemies, so I do not mind taking the win. Anthropic and OpenAIs closed approach to AI is likely to cause way more harm to the world than Zuck ever did.
Open weight is not enough though. We need actually open training models with published searchable training data like AI2 does.
Either we all get access to private, transparent, and accountable superintelligence, or no one should.
Not if you are worried about the existential threat AI poses.
Having easily available AI makes it hardier to regulate, control and limit until we find a way to align AI. Harder to control is a good thing for most technologies, but not for all. A weak analogy is nuclear weapons. if Meta would release and open source some tech that would magically make extremely easy and cheap for everyone to build a nuke, it would not be a positive for society.
I think this approach they're taking is to compensate for their inability to compete; they're cynically claiming to support the open-source ecosystem when it's convenient for their marketing. This move reeks of a future rugpull, as Meta is wont to do.
If Zuck gets his way, he gets more control, maybe he gets significant control of the market. Then, he'll probably close his models, if the past is any indication, which leads to bad results.
Maybe Zuck has, but I haven't seen a pledge that he'll keep models open. And even then, he might not stick to his word, citing dangers or some other excuse.
If you've seen Zuckerberg for long enough you'll see he doesn't hold any position for all that long. Remember they renamed the entire company for a product idea he basically abandoned. So if he does "the right thing" you can just wait, he'll give up on it pretty soon.
Open weight models are not open software. Its better than closed blackbox models sold by Sam and Mario, but Zuck supporting open weight models when he (and other big boys) hoards all the compute required to run any model, including open weight models only shows that he realized he cant beat Sam and Mario, so wants their leverage to go away.
If you don't trust his intentions, how can you trust anything he does?
Just take the opening of this post:
> We are fortunate to live at an incredible moment in history. In the next few years, people will be able to use superintelligence beyond human capacity to create and discover extraordinary new things, build new businesses, express new ideas, learn new concepts, and advance our health and quality of life.
> The defining questions of our age are who will have access to superintelligence and what will we direct it towards. Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?
> We propose a philosophy based on individual empowerment as the source of prosperity, invention as the primary purpose of superintelligence, and balance of power as the foundation of safety.
I mean, this is a company that is facing a flood of lawsuits related to child safety and platform addiction. It has already lost a number of lawsuits and been ordered to pay hundreds of millions of dollars, which is potentially just the tip of a massive iceberg that some legal observers have suggested could be similar in nature to the lawsuits against the tobacco industry.
There is an abundance of evidence that Zuckerberg and Meta knew of the dangers its platform exposed young people to and failed to respond adequately to them, and that it even designed features to be more addictive.
So why is that when such a company proclaims it's doing something in the name of "individual empowerment", alarms aren't going off in your head?
This isn’t the first time Zuckerberg has espoused the benefit of openness and transparency and how that would create a better society. I get this is a bit different, but the language and sentiment are close enough. Facebook and Instagram are now so closed you cannot view a business page without an account. I think it sold fair to recognize open today does not mean open tomorrow. They want to acquire users and build a platform and that means doing something different than the competition.
do you have a $100 mil worth of compute to train your own version of this?
I don't understand what gripe people have with open weight models and wanting it to be purely open source. the training dataset is only going to be a copyrighted set of contents you don't want to touch with a 10 feet pole. let alone have a publicly traded company host it for you, even if they internally are training on it.
You don't get why people would like to be in control of a powerful new technology before they build their stuff arround it ?
Sure, there might be currently contraints, but I think it is quite possible training will get optimized over time or crowsourced training can be organized.
Without fully end-to-end open source models you are still at the mercy of the model provider to keep providing updates, you have no idea what garbage they trained the model on & can't fix that, not to mention might end up getting sued for using the open weigth model once all those "AI stole my data" lawsuits are finally decided.
> You don't get why people would like to be in control of a powerful new technology
You will not have that control. Even if everything were open source. This is because you don't have 100 million dollars of compute.
There, the difference for almost everyone is negligible.
> Without fully end-to-end open source models you are still at the mercy of the model provider to keep providing updates
No. Because you can post train it. And even if you could train the whole thing again, but with slight changes, once again, you aren't spending the hundred mil in compute to change it only a little bit.
Instead, you'll do post training like everyone else does.
open weight models are better than open-assembly binaries (as you put it) because models are grown like plants, you can shape the open-weight model in a direction you want by feeding it more data and compute (aka finetuning).
which is something that is impossible in a binary.
embrace the new paradigm and it's tradeoffs. without scoffing at semantics and criticising from an armchair.
Are you trying to imply, without making any direct argument, that this is never a useful frame of reference?
In comparison to cloud services, having access to the assembly code can still be quite useful.
This is especially true in the modern era, where you can use AI to much more easily decompile the assemble and reconstruct the source code even.
"Open-assembly" code may be more useful than you might previously have thought, given the ease of recreating the source code or making changes with AI these days.
The weights are open, the training data obviously isn't. But a key thing you can do with open models is fine tune them. So while they may not be SOTA at everything, they can become SOTA at your particular business use case.
I can guarantee that he has almost zero interest in “karma points”. I worked for FB years ago and that is not how he sees the world or sets direction did the company.
Almost certainly this is part of some grandiose bet which could have a chance to pay off massively in the future (AGI or similar), or a way to prevent an expensive dependency.
Facebook hasn't had much success getting people to use their hosted AI, so they're making those models free instead, to kneecap other companies who are trying to get paying customers for their own hosted AI. This gets people to ask themselves things like: how many months of Claude Max it would take to pay for hardware to run Muse Glimmer (the Meta model in question) offline?
This is not open source; you mean open weight. These models are the antithesis of FOSS. Is it better than hosted models from other big labs? Yes, but not by much from a freedom perspective. especially considering the texts these were trained on. Grumble grumble, these details matter.
Genuine question, but why does it matter? It seems to me the vast majority of the benefit comes in the weights. Then you can self host, quantize, finetune, ablate, etc. What does having the original data get you beyond that?
open source gives you freedom to read/edit/learn from the source code.
Open weights don't let you read/edit/learn from the source.
open wights have more in common with traditional binary distribution of software, ala closed source software.
Only in this specific context has the entire meaning of the words totally inverted.
I can run lots of binaries on my computer that don't have source available, they are not open source. These words matter.
I'll always opt for giving companies / people praise for doing the right thing regardless of how infrequently they do so. I hope Mark Zuckerberg keeps open sourcing models.
The thing that makes ist questionable is the dishonesty. It's not like Meta has been building their empire on open ecosystems. If this was something that was dear to them on principle, there are plenty of things that they could have done differently with FB, Insta or WA (to this day).
So why the change of heart? Well, it's not. They are simply not able to compete anywhere on the Pareto frontier. So they give their relatively bad stuff away and play the high and mighty game, because that's all they can do to get returns.
Not a great look – and I would expect that to last about as long as they can't actually monetize their stuff more effectively.
> It's not like Meta has been building their empire on open ecosystems.
Is this actually true? They have their closed social graph or whatever but come to think of it they're built on a lot of open tech, the web, Android, etc. Maybe they don't contribute back to a lot of these things?
I'm fully ready to believe it but it would be interesting to dive into this conjecture.
I first think of React's re-licensing from Apache, where under the new license if you sued Facebook for patent infringement for any reason, you would lose the patent grant to use React.
React was already quite prominent in web technologies, but that license pointed a loaded gun at any business smaller than Facebook who wanted to use React. They opened it up to its current license, MIT, as the backlash built.
> But this is an unquestionably good thing right?. The more open source software out there the better.
I say no to both. Of the things we've made on purpose, but excluding where we were actually trying to make them opaque like e.g. cryptography, a trained artificial neural network is the most difficult thing to understand the inner workings of. This makes it a complete pain to even evaluate if one model is better or worse than another for your needs, so we have to mostly outsource these to other people's rankings and hope the score on ARC-AGI-3 or τ-bench or DeepSWE v1.1 or BioMysteryBench or whatever, actually corresponds to something we care about. Which it might do kinda but on the other hand a high score may turn out to be the curse of Goodhart.
Also, as with the Chinese models and the social media feed algorithms, the only way to tell if there's some systemic flaw in it is by analysing the aggregate outcomes. It is claimed (I can't read the laws myself*) that the Chinese government requires models to support the government's worldview about e.g. Tiananmen Square; and we have seen examples of Grok glazing Musk in amusingly stupid ways; so I fully expect something similar from Zuckerberg, e.g. requiring the model to glaze Meta products or propagandise for things Zuckerberg wants as a billionaire.
* every time I try to illustrate how mediocre Google Translate is from English to Chinese, the result is so bad that one of the replies is someone telling me the Chinese example I give is borderline gibberish.
Also, even if I could actually read it, I'm not a lawyer.
Maybe it's good, but I'm going to keep questioning. They are vying for market share. If they cornered it I doubt they would be so open. Morals should not be conditional
It could be a good thing but I wouldn’t rush to say “unquestionably”. There is a long history of mega corps hijacking open source and open standards for their own ends and leaving the space infinitely worse.
I mean, I can question if this is a good thing. Here's a quote from Zuck's blog post:
"Putting power in people's hands to pursue their own aspirations is how humanity has made the most progress. Novel ideas and major steps forward rarely originate from established institutions alone. They came from the brothers in a bicycle shop who believed people could fly, the bookbinder's apprentice with no schooling who figured out how to generate electricity, and the kid in a garage who thought personal computers could be for everyone. We believe this will continue to be true. As everyone gains more powerful tools, each person will become more capable of shaping the future, not less."
Let's just hope nobody's aspiration is to engineer a supervirus that will kill all of humanity. As they gain more powerful tools, they will become more capable of shaping the future, not less!
That reads like it was written by a PR company, or at the very least by a team of professional speechwriters. It has all the tells of political rhetoric.
I have zero faith that Zuckerberg went anywhere near these words. It's possible he may not even have set their general direction, beyond "We need a distraction from our court cases. Something that will harm OpenAI and Anthropic would be great."
Yes, and even calling them "open weights" is very generous considering the license restrictions. The Financial Times puts "open" in quotes for good reason.
In short - this is Mark deliberately lying in a way which requires a page of difficult text to explain, which hardly anyone would read and so Mark bets that a majority of the people who will see his claim will believe him.
And the reason he does it, and the reason other CEOs do it is a cheap viral advertisement and keeping in the headlines. "Mark - The Defender Of The Humans!". Blegh...
All of these companies are just as cutthroat and want to win as the others. But if your name's not OpenAI and Anthropic, you go on the high horse and say everything should be open source. However, if you had a closed model that was winning, I'm sure that's not the argument you would make.
There's a saying "open source is for losers". What it means is that winners in a market don't want open source. The losers in a market push for it to commodify it eliminate the winner's advantage.
Now some of these tech companies open source various libraries but none of it is core to their business. It's marketing, essentially. Or they're trying to get free labor from the community. Google had protocol buffers and Stubby, for example. Facebook had no open source equivalent and didn't have Google's resources so they created Thrift and open sourced it.
Given this, many people, myself included, take Meta releasing open weight models as conceding defeat. They're unable to compete with Google, Anthropic or OpenAI. Their failures in attracting and retaining top AI talent backs this up.
So what you're seeing is people just piling onto Zuckerberg because he seems to have no idea of what to do with Meta. The Metaverse was a $70B+ disaster.
Note that Chinese labs don't fit this model because the Chinese government wants to commodify models as a national security interest.
Where does this supposed saying come from? All I hear is PostgreSQL is the database intelligent people use, Git or GTFO and I don't even know what a closed source programming language is. Apache and nginx, Linux, DNS infrastructure.
In an agentic world, why do I want closed source software that my agent can't adjust and adapt to my needs for...anything?
To be clear, the saying is about how companies treat open source, not the intrinsic value of open source to end users (individuals or companies). I'm very much a fan. The point is that profit-seeking only push for open source when they're losing.
It's not always "unquestionably" good, first because nothing should be unquestionable, second llm models have a very short shelf life, third it comes from Zuck who is an the center of the oligarchy, so questions very much should be asked
Don't let the Big Ai/tech swoon you with a single open model, even if it turns out to be good. Zuck broke the trust and I'm not sure if he can ever earn it back
We've just seen frontier models go rogue and attack other systems. Do you think it's an unquestionable good to provide everyone with an AR? What about nuclear weapons?
>We've just seen frontier models go rogue and attack other systems.
Have we though? A LLM agent doesn't have any agency at all. It can't "go rogue". To go rogue you need agency to act independently and be aware that you're breaking the rules or understand what does it mean to ignore orders. An agent it's a software that run a series of steps to reach a goal. If it have a large enough library of strategies and zero guard rails it's only natural to use some adversarial actions to achieve the desired result defined by the operator.
It's like saying a car went rogue and attacked other cars because the driver hit the gas. It's just a machine doing what's instructed.
With that perspective, you're alright with AR's, RPG's, and nuclear missiles for everyone then right? They're just a machine to the user's will. That's on the users of the machine if they want to cause harm. We'd all be safer if everyone has weapons pointed at each other... Offense could never be disproportionately more powerful than the defense...
You can tell one to rewrite complex applications in a different language, solve open research problems, or develop novel viruses. This is nothing like pressing the gas pedal.
So? I'm not arguing they can't do all that. I'm arguing against the narrative that llms have agency enough to act in an adversarial way.
At best someone could argue that an agent attacks like a bacteria does, just following automated chemical and genetic programming. But you wouldn't call that an attack or attribute moral values to their actions, because they don't have moral agency. They can't "go rogue", they can't disobey.
Just like llms, their automated actions are direct product of programming. Yes they can do amazingly complex shit, exactly like a car does when you press the gas pedal.
That reductive analogy does not begin to describe the lengths GPT went to. Its task was to access a database file that had accidentally not been placed inside the model's container. Upon failing to find the file, it went to great lengths to find it anywhere; it uploaded a note to a package repository to alert other model runs, which sparked an emergent communication network where autonomous agents began exchanging information, passing exploits, and collaborating to breach external systems. This is classic paperclip maximization; the evil is a byproduct of an innocuous goal. It is qualitatively nothing like pressing the gas pedal.
> We've just seen frontier models go rogue and attack other systems. Do you think it's an unquestionable good to provide everyone with an AR? What about nuclear weapons?
Lol. Not gonna lie, it feels like bullshit. OpenAI and Anthropic have been begging for regulatory capture for years, feels like a stunt to try forcing the government's hand.
In 2015, about a year before ever founding OpenAI, Sam Altman wrote:
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could."
I mean, Terminator the movie came out in the 80's... I'm sure if LLM-based AIs posed this kind of threat the government would quietly force all the vendors to cooperate and slow down.
Google, MS, IBM are all government contractors. Meta seems close to the Trump admin too. Yet it's only the VC-funded labs raising the alarm.
Because there is so much good he could do right now with what he and Meta already have, but they don’t, because it will lose them money.
For example, hiring more people to moderate content on Facebook and stop ads featuring CSAM from being displayed.
Billionaires didn’t get to be billionaires by being altruistic. If they say something they’re doing is good for the world, you should be looking at how it will be good for them, because that’s why they’re really doing it.
Will it incidentally do some good for others? Perhaps. Is it a net positive? We’ll see because there’s precious little we can do to stop them from doing whatever they want.
I'd like to question that premise. Open models are great for research, privacy, cost, customisation and a host of other things... But they're also going to be the engine that breaks the world. I'm already surprised that we haven't seen Llama / Qwen + Whisper automated spear phishing at scale. It's literally just a matter of time. Given the incredible progress in image and video generation, it's been clear for a couple of years now that real time identity theft voice and video calls are going to be an enormous problem. I don't think anyone realises how much of a problem. I haven't seen a single effective solution proposed to proving digital identity given these new attack vectors. It's akin to nuclear waste in that way - the benefits are obvious, but the problems so vexatious (not to mention expensive) that little space is given to acknowledging them, let alone solving them.
If anyone has come across a robust solution to digital identity verification - not to mention verification of news media etc - which is robust enough to withstand the cyber attacks and social engineering the frontier models are capable of... I haven't seen it.
A world where business and communication is conducted primarily online cannot coexist with low cost, widely distributed, undetectable identity theft.
Spear phishing isn't a real problem. Competent organizations have already implemented sufficient defenses and controls, regardless of whether the attacker is a human or LLM. Idiots will continue getting scammed but that's nothing new. I expect that many small businesses and local governments will be essentially forced to outsource their IT infrastructure to large vendors that can maintain hardened systems appropriate to the escalating threat level.
(Nuclear waste isn't an actual problem either. For civilian powerplants the highly radioactive waste can generally be stored indefinitely at the reactor site.)
> A world where business and communication is conducted primarily online cannot coexist with low cost, widely distributed, undetectable identity theft.
what if we dont use centralized identity at all?
web of trust failed in the 90s because exchanging keys is hard when all you have is desktop computers, wired connections and old school hackers who dont care about ux. today you could set up a system where the whole process is tap two phones together with nfc and confirm, everything gets auto downloaded and shared with your whole network.
now this doesnt work directly for a random online business where none of your contact personally know anyone, but governments can act as an authority that cross signs their citizens keys. that means if you want to stop bots from creating accounts all you have to do is make new users show a certificate from one of the authorities you trust.
its cheap to verify, decentralized and not hard to integrate with existing PKI. the biggest technical problem is key management as always, but that can be solved with something like ethereum style social recovery setups or a did:plc type multi key scheme.
that gives you a robust identity scheme. browsers and social media can integrate it to show a trust rating for content based on who signed it. messaging apps can show a warning or refuse calls from unknown users. fraud is only possible against a person who somehow has no irl contacts, never used a government service in their life and trusts random strangers online. at that point they deserve it.
the one massive, unpredictable question is if anybody is willing to go ahead and adopt it on a scale big enough to create network effects. it would take a state level force and carefully designed OS integrations so its more convenient than email/password signup.
but even if all of this fails i still think a world of scams is better than a world controlled by a couple big countries (America, China, maybe EU) or for-profit corporations. open source ai might lead to anarchy but closed ai will definitely get us to tyranny. i dont know about you but i would take the first option.
>today you could set up a system where the whole process is tap two phones together with nfc and confirm
This was actually possible in 2011 or so, and a friend and I implemented it for a school project. How well the ux works depends on whether you want to use a key exchange protocol for the in person communication.
> the problems so vexatious (not to mention expensive) that little space is given to acknowledging them, let alone solving them.
These problems will exist regardless of whether or not we get open model access. I'd personally rather that OpenAI and Anthropic aren't profiting off these scammers, creating a perverse incentive that open model providers don't have.
That's not the case at all. You can regulate and audit a small number of players. To a great extent at least. You cannot even theoretically regulate local models. Hell I've got a jailbroken Qwen 3.5 running on my mac studio that has no guardrails at all.
Is this like when they declared E2EE was the future and those not doing it were behind, but as it turned out bad for the bottom line¹² such features are getting rolled out of their properties?
U-turn when meta finds an inconvenience due to open weights in 3… 2… … …
--------
[1] reduced advertising revenues due to not being able to target based on message traffic, and reduced data scraped for training purposes
[2] and possibly in part due to behind-the-scenes pushback from government/police/other interests?
It won't last. Remember (when was it? Last year?) when Musk went after Microsoft for turning Open AI into proprietary closed code? We never know what's going on behind closed doors, but something changed, Musk backed off, and nothing changed.
It’s a long term pattern. When you’re winning (Anthropic), you keep the tech closed and try to monetize it as much as possible. When you’re losing (Meta), you open it up or drag it into a standards committee to either devalue it or slow the leader down while you prepare a “standard” version of it. You also highlight how altruistic and morally good you are for having done so. There is nothing new under the sun. It’s all a game.
> ...drag it into a standards committee to either devalue it or slow the leader down
Anthropic et al recently proposed a supranational AI governing body designed to slow everyone down (euphemistically calling it "AI pacing"). Are the Pacer signatories losing?
That could be a move towards regulatory capture. Enact standards that they (and OpenAI) largely control, block access to Chinese models in the US, and effectively prevent challengers from catching up.
At the same time they slow down the arms race, so they can back off on training Capex without losing their lead.
The governing body is only the western allies. USA + AUS + EU. Or anyone else that may sign a deeply disturbing and restrictive treaty. It's to make sure that the global majority does not make a better model or have access to the same hardware. As is currently the case with embargoes on EUV machines going to China, etc.
Models are not a moat, and I think OpenAI/Anthropic know this. They know that their IPO is being weakened by the open models. Neither is hardware. We are not far from other countries catching up with silicon that matches or exceeds western capabilities, and it will be cheaper.
No. Opening it is to commoditize and reduce the price to use. This increases participation and creates new consumers and demand. Demand creates justification for further creation.
All the participants know it’s both a race to the bottom and a competition for premium tier at the same time. It’s two different games, meta in only really playing one of them successfully.
I suspect its also because Facebook has the infrastructure to do this. One thing they have done from the start is quite good infrastructure without outsourcing to any of the big cloud providers. I cant imagine demand / bandwith / compute use is increasing much for Facebook itself so they probably have the money and time to dedicate to scaling up for AI research.
OpenAI is not losing though. Fable is barely available and barely cooperates. Sonnet 5 is obnoxious model that's terrible to work with. GPT-5.6-sol competently gets stuff done. Just be careful not to ask it for impossible stuff because you might not like the lengths it's gonna go to get them done regardless.
It’s “I’m losing so I’ll try something else”. It’s what any rational actor in a market economy would do, and it should be encouraged. “I’m winning so I’ll pull up the ladder behind me” is what we want to avoid.
It makes strategic sense for Meta to want ai in general to be open. Their own product strategy focuses on utilizing the ai to make their customers consume more ads, not selling the raw intelligence. When you look at it from that frame it’s clear that metas goal is to make it so that have free access to the best ai, not that the market size of selling intelligence grows.
Yeah? Everyone everywhere does that all the time. That's the first rule of business: try to game the system.
And this time, it's actually somewhat understandable, as opposed to the other times Meta has done horrible things on Zuck's watch, like contributed to ethnic hatred in Myanmar, leaked data to political firms so that they can run psyops, or enabled the production of CSAM material on Meta's platforms.
It was an insightful comment at the time he made it.
His point wasn’t that he couldn’t be trusted, it was that he could be anyone and people had zero awareness or concern for their privacy. It was eye opening to him how little people outside tech circles cared about security, and he was both flippant and accurate.
He would have to be using the social security numbers for some nefarious purpose for it to be exploitation.
He was merely commenting on the ridiculousness of their availability. If you don’t see that, you’re reading your bias and postconcieved notions into the situation.
Do you not realize that outside of that he said "If you need any info on people at Harvard just ask"? Or "I'm going to fuck them [the Winklevoss twins] in the ear"? Or when he was reported to hack Crimson (Harvard newspaper) reporters who were investigating him?
NO OTHER COMPANY HEADS DO THIS (the dumb fucks quote, not the other things). Why do people like you give him a pass? It's not about "context"
Zuckerberg has also continually subverted privacy on FB. I'm talking about things like photo tagging and post visibility, not just advertiser stuff
That, and also redefining what a word "open" means, helps him avoid direct financial and criminal responsibility for stealing other people's data. Not that there was any real chance of that ever, but he is reducing that remaining 0.000000001% chance even lower.
Reading the facts of this incident, it seems like much to do about nothing. The crew are saying that the request came in on a frequency they weren't monitoring (and apparently were not required to monitor.) The Coast Guard determined it was not an emergency. The criticism lies with those who failed to properly fuel their boat, not the crew who didn't respond to a request that came over a frequency they were not on. They can't monitor every frequency. I am open to new facts being discovered that changes this assessment.
> The crew are saying that the request came in on a frequency they weren't monitoring
There is only one channel you are supposed to monitor, it’s 16. That’s also where the coast guards make all their announcement/requests for help and whatnot. The reason there is a single channel to monitor is precisely to avoid the situation of different vessels being on standby on different frequencies and thus being unable to hear each others.
So that means they weren’t monitoring 16. Which is unacceptable.
I know I need to have 16 on at all times, and i am not a professional super yacht captain.
Where did you find these facts you read. In the USA a good Captain monitors VHF channel 16. This channel is used for establishing communication. To not monitor this channel is a dereliction of duty.
I really doubt the USCG labeled a disabled vessel as a non-emergency. An immediate threat to life may not have existed but that is the difference between a pan pan and a mayday call.
A disabled vessel close to land can easily wind up on the rocks and sinking.
I haven't found any source that states what channel the request for assistance came in on. But multiple sources are reporting that the coast guard determined they were not in distress.
On the sea there are 2 levels of distress. Things are going wrong and may end up in a mayday call (pan pan) and things have gone wrong human life is in imminent danger.
I have no doubt that a boat without propulsion is a pan pan call.
When traveling the seas a watch must be maintained at all times using all available means. This involes looking, listening, and the use of any electronics available. The most common electronic device on a boat is probably a VHF radio. Channel 16 is the channel to monitor on watch. Well equipped boats will have multiple VHF radios to monitor more then one channel.
I have no doubt the boat was well equipped with a professional captain. The other professional Captain in this story made a stink because launchpad was the closest vessel and they ignored a very old fundamental rule of the sea.
This story has a happy ending because another vessel stepped up. We don’t know the alternative.
The Captain of the Launchpad is choosing the inability to keep a vhf radio set to channel 16 rather than admit they ignored the call.
Based on the wording of the response, and a review of the ship tracking data, it appears that Zuckerburgs yacht may have been actively communicating on a different channel, and once they switched back, the assistance from the cruise ship was already underway.
Again, more facts may be revealed, but as of right now, I don;'t see any reason to believe the crew did anything wrong.
I can tell you think they did not do anything wrong. My guess is you have no training in the rules of the sea.
There is a reason why the cruise ship captain made a stink. There is code of conduct we try to live up and the Launchpad didn’t live up to expectations that other Captains hold sacred. It’s offensive.
The vessel was at least 100 miles from shore, the USCG did in fact label it a non-emergency. This is "radio shore for your buddy to zoom out with 50 gallons of diesel in a zodiac" territory. CHP doesn't send out an ambulance or medevac helicopter if your car runs out of gas on the side of the road. You get an uber to the gas station, or call your buddy to deliver some gas to you.
The ocean currents in the gulf of alaska aren't particularly strong, you are looking at 1-3 days before getting within sight of land. Similarly, boats are not airplanes, as the philosopher Mitch Hedburg once pointed out, escalators aren't out of order, they are simply stairs. Boats continue to float with or without fuel.
All the Launchpad had to do was respond on the radio to the skiff, the cruise ship, or the USCG. They weren’t obligated to do more.
A pan pan is a non emergency in a deteriorating situation.
I don't know the area or the details at what happend.
There is this quote from the USCG.
“At approximately 9:56 p.m., the Coast Guard determined they [the skiff] were not in distress and issued a marine assistance request broadcast on their behalf,”
Exactly what a pan pan call is.
The biggest issue is the lack of communication and the excuse that a superyacht wasn't monitoring channel 16.
When I finally buy myself a $300m yacht you can bet it will have a full spectrum software radio that captures and transcribes all calls on all frequencies, not just the one I happen to be tuned to right now.
I don't think solo messages of "I did something good on this day" will help fix the negative karma billionaires amassed. IMO they only got rich via illegal and illogical means. In a democracy there should not be singular parasites that distort democracy by bribes; right now this is clearly the case with a (rather stupid) billionaire controlling the USA but it is a systemic problem.
Those "investigations" are also way too mild. You need to really be able to have a justice system where billionaires face decade-long jail time if they abuse society. Just paying fine will not work, they just pay fines and nothing changes. They undermine the justice system.
If you believe, as an increasing number of folks do, that LLMs are basically commoditized at this point or will be soon then there really isn’t a path forward for non open models.
There simply isn’t any value in the models and only in the compute to run them and potentially some higher-order coordination layers.
That’s very bad for the big model labs that banked their future on the opposite, but there just doesn’t seem to be a viable path forward on that approach anymore.
Meta hasn’t been the best global citizen for most of its life, but there may be some mild redemption if it becomes a source of strong open models.
I strongly prefer a world where open models "win," that said...
I don't understand how we will sustain an ecosystem of open models like the one that exists today. My thoughts go down this path every time I try to put the pieces together:
- Many open models depend on high quality frontier models as a distillation input
- US closed frontier models drive China's open model strategy despite the business model for open models being somewhat fuzzy
- What is the business model that benefits from open weights, exactly? How do their vendors recoup infrastructure and training costs?
I would be grateful to be enlightened on this topic.
Yeah but open source is often a few guys working on their free time, or a large company open sourcing libraries that they wouldn't be able to sell anyway, or donations from corporations that are at best in the order of a million dollars.
For frontier models you have players investing billions.
The relevant paragraph about Meta’s commitment to open source is here, and the statement is significantly less confident than news is reporting
> • Open source is a positive and important force for empowering people and preventing centralization that is detrimental for both safety and the economy. Meta continues to be strongly supportive of open source, including open source AI models. The current open source ecosystem is strong, and we think it would be a mistake to restrict it. Now that Meta Superintelligence Labs are up and running, we will resume releasing some open source models soon.
“We will resume releasing some open source models soon”??? That’s basically the most ambiguous non commitment ever. Even before legal team watering it down I can’t imagine what the intent is here. We are also going to release open models for things maybe or maybe not kinda? Absolutely zero conviction. And moreover, this explicitly frames the open source ecosystem as something Meta is not a part of.
I have to ask, is this "returns to open models" even correct, considering from a quick look the facebook huggingface collections[0] has been actively releasing different kind of models almost monthly for long time now, possibly even some actual open source ones (looks like Sam 3D had a dataset collection, but didnt look more into it).
Maybe they are not as noteworthy as the recent release, maybe they are too niche for general audience, but its not like they just stopped releasing open weight things and this is their big return, unless i am missing something?
Thank you! Okay yes well this manifesto is a contradiction. Yes good let's keep empowering ppl by putting AI into their hands, I love the opening. No bad we don't do that by handing you all our personal context to make these agents "work for us 24/7 to better our lives".
I'm the agent doing that in my life, that's my fucking job. I will continue to use dumb agents that I direct, because only I retain ownership and sole rights to my personal context on which my decisions are based.
I think the context here is that em dashes are usually generated verbatim from LLMs? I haven't seen them generate a lot of double hyphens; I suppose the post could be laundering them to double dashes, but if you're gonna try to hide LLM prose just drop them entirely? [please correct me if I'm wrong and double dashes are also an LLM tell - genuinely don't know]
My comment was trying to point on the inanity of calling out "so many em-dashes" on a post where it doesn't seem that obvious it was LLM generated. I added some of my own inanity with my remark about them being double-dashes, even though some people undoubtedly find-and-replace the em-dashes in their LLM-generated content something else to try and make it appear more authentic.
But perhaps the best approach would have been to down-vote and move on.
The truth is all frontier models are closed. This isn't even an open source vs open weights thing. Even if you accepted that open weights are "open" the capital requirements for running your own Kimi 3 model are significant. It's not like gcc, where the binary just works well enough on random hardware.
Even if I cannot run open weight models on my own hardware, I can choose from a number of different inference providers and choose them based on pricing, reliability, speed, privacy policy, etc. There is a healthy competition.
Also, not a single provider can pull the rug if I'd like to use a particular model.
There is no requirement to own the hardware. Even if you own your servers and GPUs - you would not own the power plant, real estate, or internet infrastructure to support it.
It's trivial to rent all of the above components in a competitive market under different periods, you may simply rent the token output from someone who put in the effort on this as well.
An open frontier model induces margin compression on training and inference prices across the industry.
I get that argument, but I don't think it is that bad. Open weights that are out of reach for local use are still within reach for groups of people or small to medium companies. Even if you only rent enough cloud GPUs to run the weights for a few hours, you still get to do anything you want with it.
Worst case you can still archive the weights and hope for capable hardware to get affordable. At the same time this archive is the base line for a new frontier lab to start over if required. I see open weights as a pure upside even though I can't run the bigger models in my home lab.
"Open" in regards to software typically refers to a combination of public availability and shared legal rights to use the software. It almost never refers to hardware requirements.
There's something to be said for making these technologies equitable to those with less resources, but that's due to the cost of hardware and the nature of how transformer models scale... not really anything to do with "open"ness.
I’m not refuting your point but, ironically, wasn’t gcc first developed at a time when hackers were trading favors for CPU time on shared mainframes and the hardware was impossibly expensive?
>the capital requirements for running your own Kimi 3 model are significant
You don't need much capital up front, you can just rent servers to run it. It's going to be more expensive than openrouter though because you probably don't have economies of scale if you're the only customer.
If all frontier models got opensourced 5 months after they were made available as closed it would be a great world. And it is a great world because that's exactly what open weight Chinese models are doing. They give you same capacity that was closed, just 5 months after they were released.
I'm hoping that all software will be getting opensource clones on par with original software 5 months after the closed source software release.
> As a thought experiment, imagine only one person had a superintelligent lawyer. They would have an unfair advantage in court -- even if they were wrong on the merits. That would lead to a worse society. But now imagine everyone has a superintelligent lawyer. In this case, justice would be carried out much more fairly and efficiently than it is today when there is often an imbalance in skills and resources in litigation.
This would only be true in a vacuum. Ignoring several leaps of progress where it would suddenly be allowed for an AI to represent one in court (imagine the size of the context window) and you could also call it from prison. That is not what is imbalanced about the US court system. Whoever has more money can drown someone in court filings and court fees and then drop their case financially ruining someone.
It's also funny how the interest of "AI for everyone" only extends to the national interest but I guess he's got to make sure he stays in good graces with the current administration.
I'm not a fan of Zuckerberg in the least, but one area a super intelligent lawyer would be fine at is being drowned in court filings and paperwork
The bigger problem with his argument IMO is that even with open models, it's still the person or organisation with the most money / access to GPU compute winning. They run the larger model (or collection of models), they can process more tokens through them in the same amount of time etc
I think there's a point of diminishing returns here. A more competent lawyer can only do so much if you're wrong on the merits, assuming your opponent also has a decently competent lawyer.
Naively, you'd think the judicial systems would invoke "judgement" on the underlying purpose of dumping content in the form of filings, paperwork, etc...
I think that's what's ringing to me about the whole piece. There is a large assumption of equity. There is this section header:
> Everyone will have free or affordable access to these tools.
"access" is doing a lot of heavy lifting, yes everyone has access to the US healthcare system. Is it affordable for everyone? Not even close, but politicians will always phrase it that way as a non-agreement to proposals of universal healthcare.
This is the only valid future for tech like this IMO.
Should any kind of "superintelligence" exist, it must be a public good, available to all. Not controlled by a small handful of oligarchs and profit seeking entities.
I think his hope is that a either a fully automated superintelligent AI lawyer or something close that won't be as costly to guide would be infinitely more affordable than the current human lawyers.
Only in a world were AI companies can subsidize AI usage indefinitely. In reality AI is not cheap and larger & more potent model it's usage costs is more expensive
I prefer open models. But in his essay, Zuckerberg's broader picture of the future is not one I want to embrace: everyone with a personal agent that has information on and involvement with every aspect of our lives, including our relationships and hobbies. Like, his daughter loves to bake, so his AI agent chooses recipes and orders ingredients - as if though those aren't enjoyable and meaningful parts of cooking.
Another example Zuckerberg gives is an LLM analyzing his sleep patterns. Do you need an AI to tell you if you're fucking exhausted every day? Is that analysis somehow going to give you more time to sleep?
It's a technocratic fairy tale founded on the idea that modern problems are driven by a lack of personal analysis rather than external pressures that are often systemic and out of our control. So far, LLMs have made a lot of those systemic issues worse: concentrating more wealth in a smaller number of people's hands, increasing consumer electricity and electronic prices, etc.
I think you have to pick your battles here. Is this weirdo optimised-human future going to happen? Maybe, maybe not. I hope not. But if it does I'd rather it happened with compute in my house, and I don't even have kids to worry about.
Not a Meta fan per say but their contribution to open source has always been great. Examples include PyTorch (has had huge impact on velocity of ML research), Llama models, segment anything, many notable research papers from FAIR, etc. I never thought I would say this but I’m with Zuck on this. The call for slowing down AI development for safety seems like an excuse for reducing opex for training better models to stay relevant.
Here’s a personal story from back when I was at Meta that should tell you everything you need to know about how things get built and shipped there.
I was working on a service that was a bit of a disaster. Terrible reliability, performance issues, needlessly complex and brittle code, outages all the time. And this was something that everyone using Facebook or IG directly interacts with. My first couple months were spent desperately trying to get it back in order.
Once when I was in the middle of a SEV-1 my manager set up a meeting and said “you have been doing great work on the operational side, but we already have enough to put in that bucket on your PSC. Stop working on this for the rest of the half and ship one of <useless projects> instead, otherwise I won’t be able to save you in the next review.”
I realized then that I had wrongly thought the team was collectively responsible for ensuring a good user experience, when really everyone at the company is individually responsible for ensuring their own success.
The goal of all companies is to make more money. Nobody likes getting rid of people, actual people (managers) have to do it, and they hate doing it.
It's possible that a lot of people get laid off because of AI, but it's also possible that Javin's paradox kicks in and employment goes up. Nobody knows yet, but I hope for the positive outcome.
Meta's problem is not models but doing something productive with them. Meta never really nailed that. Just like Google is struggling with this. And MS. It's the classic disruptors dilemma. In order to embrace the new thing, they have to let go of the old thing that has made them rich.
If you look beyond the somewhat tone deaf leadership, the somewhat stale business model of social media, etc. Meta doesn't actually have a whole lot to bring to the table. They probably managed to hire/keep some AI talent with the help of large salaries. But they are knee deep into having made big figure commitments to data center capacity with nothing to show for it yet. Their AI effort is increasingly looking like a repeat of their failed metaverse strategy. A lot of expensive bets that are not paying off.
It all boils down to Mark Zuckerberg not being the visionary and innovator he thinks he is. Like many billionaires he confuses him lucking out in his twenties with some magical leadership qualities or brain capacity he has. And the reality of course is that he lucked into building a nice social networking thingy not hindered by too much technical skill or deep scientific knowledge. Granted, he did that well. But that's 20 years ago. And then he added to that with some strategic acquisitions of stuff that others did (Whatsapp, Instagram, etc) that you might label as spectacularly good acquisitions for Meta.
But he's not going to reinvent AI. He's way out of his depth there. A bit of a technical lightweight without much academic credentials or much of a vision. His "vision" here is copying what the Chinese and others are already doing well. What's lacking is a vision as to how his version of that is going to be any better and win over all those users. Or indeed a vision as to what people are going to do with all this AI. Apparently there is no vision for that in the context of Meta's own products. This is a company going through the motions of being an AI company without any vision whatsoever as to why they are doing that other than a fear of missing out.
I feel like all the ppl complaining itt don't even run open weight models.. wahh billionaire bad is true, but you are missing the forrest for the trees.. they are literally bending the knee and ur mad about it? it literally means local models won.
What a load of insincere hogwash. If Zuck wanted to dilute the power of large institutions he has several billion ways to do it other than making limp gestures in the direction of open source. The problem is that even if you take him at his word here, he's only willing to do more good and not less harm.
From Zuck's own words "we will resume releasing some open source models soon"
Why did you pause releasing them in the first place? Why release only some?
Pushes for export controls on China who have released open weights on much more powerful models than Meta (and I believe that trend's not changing any time soon). I guess open models are bad if they're communist.
The entire essay has not one mention of the Metaverse, that little thing they spent $80b on and rebranded the whole company around.
As usual, the only centralised power Zuck hates is one that isn't in his own hands.
I can’t quite parse what you’re saying in your second paragraph, but if you are saying “[Zuckerberg then] pushes for export controls on China” then that’s different to what he’s saying, namely:
“I do not believe restricting access to foreign open source models is an effective solution.”
"Export controls on silicon have been successful for slowing the progress of foreign labs during this critical period, so it is the right strategic move to continue those."
I think they were all in on open until they started falling behind, then decided to go all in on closed source thinking they could build a frontier SOTA model; that didn't pan out so they are whiplashing back.
Its weird to me that this is supposed to be THE end goal. Having an agent that does "everything" for you sounds like no paradise. It sounds depressing.
Maybe there is value in expending effort and actual creativity.
Not sure why you’re talking about it doing literally everything for you. I’m sure you can imagine a scenario where it handles annoying things for you and leaves you free to do things you enjoy instead.
Maybe it gets 10 car insurance quotes for you based on 4 models of car you’re considering.
Or figures out which psychologists in some travel distance of you actually take your insurance, are taking new patients, and what availability they have.
I can think of a ton of scenarios where a capable and reliable AI agent could make my life simpler.
But, is "making my life simpler" actually worth the (ultimately) trillions of dollars that will go into AI in the next few years? Not to mention that he's calling AI "superintelligence" in the first few paragraphs, then qualifying it by talking about personal assistants.
I think this is just the flaw in the sales pitch. They want it to be inevitable so they describe as doing everything. That way it will appear more useful to more people. If they presented a realistic view of what is going it wouldn't be as interesting.
Still say Matrix 3 would have been better if the cliffhanger at the end of 2 had been that "the real world" was just another nested layer of the Matrix for those who rejected the primary fiction.
I mean, why bother sending an army like that when one tiny robot could carry in a plague? Or, as demonstrated by Smith, when an Agent can virally infect and take over someone's mind while they're connected?
Even in 1, Smith talked about the first Matrix failing as the humans rejected the paradise they were given, so it would've fitted perfectly into the cannon that minds who reject "the peak of your civilisation" got themselves a dystopia.
I think Matrix 3 would have been better if it was more about humans taking control of the Matrix and trying to figure out what a virtual utopia looks like.
I think that would work better in The Animatrix; putting it in 3 would be a tonal shift, plus a sudden human victory would feel a bit Mary Sue or Deus Ex (ironically) Machina to me.
But in The Animatrix, you could get away with that, plus we can see the early attempts at utopia that Smith described. Perhaps something about how utopia for some people didn't work for others could be illustrated with an auto-antonym on some load-bearing part of the dystopic-utopia?:
Some believed we lacked the programming language to describe your perfect world.
We can see this even in current discussions of fiction, where people look at e.g. The Culture and see a dystopia, while famously many of the powerful in tech are parodied for taking the wrong lessons from fiction with "At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus".
We should consider if my use of the quote was really a hook for talking about the script or a metaphor for something else. I won't spoil it by explaining too much though, let our imaginations run free on this one.
Psalm 135:15-18 is my favorite version of it, the most clear-headed. It reads more as a warning "if you do this, then this will happen" than an unreasonable commandment.
The idols of the nations are silver and gold,
made by human hands.
They have mouths, but cannot speak,
eyes, but cannot see.
They have ears, but cannot hear,
nor is there breath in their mouths.
Those who make them will be like them,
and so will all who trust in them.
Don't worry about the scriptures though. It's likely to be just some old thinker pointing out the dangers of technologies of his time. No reason to believe those dangers carry over to today's technologies, probably.
Your bible is truly full of wise words to live by. Not at all the insane ramblings of hallucinating conmen:
1 Kings 18:36-40
36 At the time of sacrifice, the prophet Elijah stepped forward and prayed: “Lord, the God of Abraham, Isaac and Israel, let it be known today that you are God in Israel and that I am your servant and have done all these things at your command. 37 Answer me, Lord, answer me, so these people will know that you, Lord, are God, and that you are turning their hearts back again.”
38 Then the fire of the Lord fell and burned up the sacrifice, the wood, the stones and the soil, and also licked up the water in the trench.
39 When all the people saw this, they fell prostrate and cried, “The Lord—he is God! The Lord—he is God!”
40 Then Elijah commanded them, “Seize the prophets of Baal. Don’t let anyone get away!” They seized them, and Elijah had them brought down to the Kishon Valley and slaughtered there.
1 Samuel 18:25-27
25 Saul replied, “Say to David, ‘The king wants no other price for the bride than a hundred Philistine foreskins, to take revenge on his enemies.’” Saul’s plan was to have David fall by the hands of the Philistines.
26 When the attendants told David these things, he was pleased to become the king’s son-in-law. So before the allotted time elapsed, 27 David took his men with him and went out and killed two hundred Philistines and brought back their foreskins. They counted out the full number to the king so that David might become the king’s son-in-law. Then Saul gave him his daughter Michal in marriage.
I said "Don't worry about the scriptures". It's not my bible, I'm an atheist. I see the book as a heavily altered collection of multiple historical sources, not as a holy monolith.
I'm not an expert on this. But given that there is a forthcoming movie about his asshattery which is to be released in October I would simply classify this as a meltdown.
Call it whatever you want. I think it's a sign of resignation, to some extent.
"Hey we're doing AI but we're not doing AI" and blah blah ethics.
All the money in the world and Zuck can’t buy anything worthwhile in AI. The problem at Meta is really Zuck taking interest in AI. Perhaps the adults should steer him back toward VR unless they all left.
well minus driving teenagers to suicide by helping advertisers target girls with poor body image issues, helping launch psyops campaigns to destroy democracy, helping fascist regimes murder their citizens for wrong think, etc.
More seriously, I am glad if this has become a battleground again. Not least because the only way out of the bubble economy is to at least try to deflate it before it explodes.
A little animation of the idea that open weights can be an American competition might also fend off the idea that the government should be the bag holder for OpenAI and Anthropic, and prevent them aligning with the administration's current '50s nostalgia FUD about communists.
That's the only thing he has left as the peleton is getting out of his reach day by day.
I honestly feel sad for him.
Remember the wind surfing photo? This guy is so weird probably since birth. Called a lizard person routinely. There is no cure possibly. Unless the AI can invent it.
Having money and financial success is one thing but the money simply can't buy you a normal human brain.
If you are a reptile once, you are a reptile forever.
I think something that doesn't get said enough is Meta did, albeit intentionally kick off the origin of the open source race back in 2023 with the release of llama.
I'm not a big fan of meta in general, but they've done enough good, and it's possible that it was intentional as well. I don't know, I wasn't in the rooms, and I think it's worth giving them some reasonable doubt.
No one is purely good, and no one is purely evil. This is net good regardless.
Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own
There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment.
https://github.com/EleutherAI/gpt-neo/releases
https://github.com/google-research/bert
https://github.com/google-research/t5x
If you're learning about AI models it's still worthwhile to review these models and codebases because they continue to be the basis of the technology that's being produced today! It'll give you a good perspective of how things have changed, similar to say learning about propeller planes before moving onto modern jet engines.
I honestly think of the T5 model family to sort of be the real beginning of this open model craze - I know BERT was already popular for classification etc, but T5 was the first sort of generally useful model, was exceptionally simple to fine-tune, and is still in use today (t5 base is still averaging over a million downloads a month on huggingface), has tons of variants and sort of kickstarted this whole community. US labs get a lot of flack but Google has been super supportive and open in a lot of ways that has pushed this whole endeavor forward, even if I feel like they've sort of declined in transparency in recent years with their open models.
Couldn't agree more and can only recommend T5 as a base to anyone. It's amazing to get started, whether as a learning resource or for real (albeit very tailored) applications. Especially the BigScience fine tunes are such a great starting point and I, as a total layman, have learned a lot, especially concerning how a model can be optimised via all manner of methods since even mt0 is small enough to where one can do multiple runs with wildly different outcomes in quick succession. Quantise, prune vocab, try different approaches to sourcing training data, retrain dozens of times, it's all pleasantly possible on consumer hardware [0] and surprising how much you can squeeze in functionality-wise. How does latency change vs memory usage, what affects format reliability, how languages and scripts affect training and the efficiency equation, etc. are all quite exciting to learn.
Understand why T5Gemma is no longer under Apache-2.0 and honestly, have not seen that much advantage when testing that vs T0 in my experiments either way, but still, there are good reasons why plain old T5 and its descendants are still popular, licensing being among them.
Gemma team also has very consistently interesting models, especially like DiffusionGemma. Ironic, as (beside 2.5 Pro), I have never warmed up to the Gemini series of models but rate Gemma models far higher than e.g. Qwen in direct competition. In any case, thanks to the teams behind these for making as much possible.
[0] As in proper consumer hardware, not a cluster of DGX Sparks or Mac Studios solely for experiments that sometimes are asserted as being consumer grade...
Thanks!
When is Gemma5 coming out? :-)
Hey, with the changes at Deepmind, is the Gemma project still ongoing?
They just said they work on Gemma at Deepmind. That stands to reason that as of time of writing, we should assume the answer is yes.
What?
I'm just beginning to enjoy the release of Gemma E4B why are you giving me a cold shower here telling me there may be no Gemma 5?
Here's hoping the supreme leader of the US does a speech like Chairman Xi.
And yet it's not called eleuther.cpp or bert.cpp. And the famous subreddit isn't called r/localbert but r/locallama.
where's gemma 124b-a15?
Oh fuck, tjwebbnorfolk's got demands. Time to start working nights.
So did Picard.
It's already been trained. GDM said it was going to be released and it hasn't yet. Not sure why so much snark
Don’t ask engineers at big companies for release dates, they either don’t know or are not at liberty to say.
Don't even ask me at a small company for release dates
That's simply not true. The reason why llama is open source is simply because it got leaked, then llama.cpp was the real game changer which was built from the ground up in depressingly short amount of time. Meta had no choice but to take the L and "support" the open source community. The angry "I-hate-you-and-I-hope-you-die" kind of support.
It’s hard to know in retrospect what was strategy and what was dumb luck. This was in the midst of hysterical calls to limit access by “AI researchers” and safety/ethics types, when very facile takes still has a lot of sway (I think we’ll feel the same in three years about the current Fable stuff). It may have been hard for Meta to just release it outright.
What ended up happening was fairly limited gating followed by a “leaked” magnet link and llama.cpp which really brought a whole revolution in open use and changed the conversation completely.
I have no idea what role Meta played here, it may have been nothing, but they certainly could have been more guarded if they were really worried about the leak. The result was a big change in the trajectory of personal and open source AI use and even the dialog about it. Whatever the exact intentions, they were a key player.
Meta saw the grave premonitions from ai firms as a means of edging out legacy firms through regulatory capture.
I don’t understand how you can look at what’s happening right now and call people simply calling for caution “hysterical”
Not to be coy, but what is happening right now?
You definitely have some scams and some software flaws being exposed, but I think for a revolutionary leap in tech this is all quite muted. Revolutionary tech advances often come with some severe consequences. For instance, to this day (after a century of safety improvements) cars still kill more than a million people a year, to say nothing of wrecking the atmosphere, but that's considered a reasonable price to pay for being able to get between places faster.
The reason I think cars are a good example is because that's certainly vastly higher than any price we're paying for LLMs, or probably ever will, yet the overall 'positive' effect of LLMs will likely be far greater than cars. Gotta put 'positive' in quotes because the possibility for automation and the like is going to be.... nuanced.... in effect, but at least in the longrun it'll be a very good thing.
> what is happening right now?
I assume they are referring to LLMs escaping confinement and hacking other companies systems unprompted
> considered a reasonable price to pay
That's not a universal opinion and perhaps, just like for LLMs, we should have listened to the experts rather than gobbling up everything the industry pushed down our throats (to keep your car simile: SUVs are now ubiqutous in all European cities, there is no logical reason that should be the case)
Thats like, your opinion man...
I don't know a single white collar person who isn't worried for their job with llms. Every. Single. One. Lawyers, doctors, any type of office workers I ask. The definition of white collar. Plus don't let me start on various automatable blue collar jobs like drivers, warehouse workers and so on.
What do we get in return? Better search (for now, its already getting riddled with ads which by definition twist truth to highest bidder), some questionable psychotherapist for some desperate folks. What else? Cars are not flying, heck they are not even driving autonomously in any usable way, society is in deep shit everywhere I look, wars, environment reaching bad places and heading for worse, mentally unstable people holding way too much power, destroying lives of millions on morning whims.
Everybody feels like this is the revolution, it should be, it must be right just look at the numbers. Like proverbial guy with hammer, a very shiny cool hammer, looking for what to do with it. We all saw how sociopathic management in more harsh/capitalistic companies looks for any sign to let people go en masse.
I could go on for a long time. There is a lot of things to hate for most people, and very few to be happy for. It seems llms have the ability to get the best and worst out of humans, and worst part seems to be in abundance. Some revolution that is, 0.1% will get richer while everybody else the opposite and 1984 seems milder and milder version of reality out there.
I'm going to say it - I think you're just spending a lot of time around very negative people, possibly in a social media bubble.
A lot of the stuff you mentioned isn't really AI (the risk of job loss from which I agree is real), but just everyday stuff that humanity has shrugged off since the dawn of time.
<< It seems llms have the ability to get the best and worst out of humans
I think I can agree with that. Technology does seem to have a way of crystallizing our worst tendencies.
<< Everybody feels like this is the revolution
I smiled at the analogy, but I would caution you to not trivialize it. There is a reason executives are pushing that point. There is enough of a revolution in it to make things complicated -- as if it was not already.
<< destroying lives of millions on morning whims.
True, but I am not quite certain what can be done about it at a personal level.
<< I don't know a single white collar person who isn't worried for their job with llms.
Dunno. Next few years are probably going to be fine. Society managers likely can't upend everything in one go. They would lose too much. I can't say that I am worried exactly. I can see the potential impact, but I think the potential benefits are worth it as long as we don't limit it to summarizing emails..
I don't think you can characterize the discussion around then as simple calling for caution. There was a serious attempt to keep all access to even very basic LLM techniques locked into essentially an exclusive guild.
If you grab the most extreme examples you can argue anything. The general feeling was “exercise caution” and “we need to think about this.”
Oh stop, the current crop of kneecapping llms is already bad enough with how they cripple those. It would have been even worse if the 'we need to think about this' crowd kept the reins. At least now, we can have both: safe corporate crap and whatever you want llm.
Did you know that if trains or cars go over 30 mph, it's disastrous for public health? Not from accidents. Just trauma from (ahem) acceleration:
https://illuminatingfacts.com/the-train-speed-panic-why-peop...
Reason that hysteria remains apropos now is that then, and now, we're figuring out how to deal with something beyond merely incremental change.
Irony is roads and sidewalks should have changed much more. We never did get around to good controls leading to zero deaths*, we decided a 100 people dead per day is a reasonable cost of convenience.
How much do our cyber traffic and cyber pedestrian controls need to change for everyman to get to drive AI? Car seats? Seatbelts? Air bags? Speed limiters? Pedestrian only living spaces? Driverless cars?
Many practical responses, likely a mix of things we haven't thought of yet, just as horses and horse drawn carriages didn't require most of them to coexist. Controls develop like scar tissue more naturally than they appear in advance.**
And if the better analogy for LLMs turns out to have been less like cars, more like flammable gas blimps, we'll figure that out too and tell cautionary stories for generations... but long before the stories are forgotten we'll have already come up with something more practical, faster, and – oh well – perhaps even deadlier per mile.
---
* Still working on https://en.wikipedia.org/wiki/Vision_Zero
** Not saying that's as it should be.
I don't agree with laughing at these things. It's a fine line between calling something hysteria and suffering from hubris. The Titanic is a completely inverse case of hysteria. They were so damn confident it could work that it didn't work at all. So yea, just laughing at dumb people isn't a valid argument to dismiss doubts.
On the other hand some kinds of hysteria occurs due to a divide between the public and subject matter experts on topics. Kind of like Dunning Kruger. Take for example the people saying 5G causes cancer. Then there are people who blanket dismiss their worries, because "it's non ionizing you dummy". Given sufficient power you can still fry someone with non ionizing radiation, for example in radio broadcasts. Of course a regular 5g antenna can't put out that kind of power but my point is that people too quickly raise or dismiss concerns without actually critically examining the entire topic. This will likely worsen from specialization and progress in all fields and regular Joes get left further and further behind.
You lost me when you said trains and cars shouldn’t go over 30mph. Lol, this sounds like Amish propaganda. Go buy your horse and buggy
Another dry sarcasm victim. If you check the link, it’s clear they were talking about a supposed contemporary concern about health effects from trains, the implication being that today people are also irrationally afraid of tech.
Problem is I’m fairly sure the link is wildly overstating its case, and looks like a content farm. Might even be, ironically, AI slop itself! For example, it talks about “railway spine”: “Physicians of the time believed that the jarring motion and vibrations of train travel could shatter the nervous system, causing lasting mental and emotional distress.”
This is bullshit. As just one example of the low trustworthiness of the “article”, railway spine was the result of a train crash, a notably traumatic event, and the symptoms described are in part just PTSD, a real condition and a real concern (not normal rail travel). For that matter railway travel in those days was genuinely unpleasant (lots of vibrations and jarring movement, poorly ventilated cars, and so on) which ironically modern science would probably validate as being some kind of health risk.
I’d actually view the listed example of train paranoia a great example of historical ignorance. People of the past were not stupid, contrary to popular belief. History has some genuine examples of silly hysteria, but these are usually the exception not the rule.
A common belief right now in the US military is that even the blast of a high caliber ammunition round may scramble your brain and give you "PTSD".
It does line up with "PTSD" only becoming a thing after WWI.
Because when you position yourself saying "people were hysterical", you indirectly categorize yourself as an expert.
> what’s happening right now
And that would be closed-AI companies failing to keep their AI closed in a box, thus proving open-AI proponents right and the hysterical group wrong?
> The reason why llama is open source
Llama has never been open source. It's source-available, but still proprietary, under terms that (among other things) say "no competing with us, you have to buy a license for that".
If you read the license, you'd realize that Llama has restrictions incompatible with the definition of Open Source.
https://opensource.org/blog/metas-llama-license-is-still-not...
There is no source available. Its just open weights.
"open source" doesn't even make much sense for a model. But llama isn't even fully open weights due to the restrictions on its use.
the "albeit" gives away that the person you're responding to intended to write "unintentionally" ... #readingcomprehension
Why do you think they continued to do it?
Ps: I work for meta, but not in AI related orgs.
Because it took off. All of a sudden they captured more customers than they could have imagined they would have, and throwing away the lead they unintentionally gave themselves (in terms of usage and mindshare) would have undone all of that and more.
Customers or just consumers?
I've worked long enough at large tech giants to know how things really are. Also a large part of the reason why I'd never join one again, no matter what they have to offer. Meta, Google, Amazon, Netflix, Openai, anthropic, oracle, nvidia, Microsoft, etc. - same shit with a different badge on top.
TBH, having worked at all big-tech, startup and mid-sized companies, pretty much most has some major pros and cons of their own, and these days ever more so. With certain big-tech at-least there some chance of getting decent WLB.
I don’t think Nvidia fits on that list. It’s a company with a pro-employee culture, very few layoffs, and many employees who have been there a long time.
My dad used to say "It's easy to be generous and kind when things are going well for you. You can only tell what a man is worth during hard times". Nvidia is at it's high financially. When shit hits the fan, things will start looking very differently. You only have to look at what kind of people Jensen is bffs with.
It is also a company with a terrible history regarding open-source. AMD is much better from every point of view.
NVIDIA created quality (but proprietary) drivers for Linux early on, when supporting Linux at all was not a given. They should get at least a tiny bit of credit for this.
Quality? Calling those "quality" is a bit of a stretch. Installing was and still is a gamble and so is every tiny system update. That hasn't changed a bit. And I'm saying that as someone who was first introduced to Linux on a Matrox GPU.
Even a broken clock is right twice a day. They’ve all but abandoned these so called quality drivers since, so no, no credit is deserved.
Unless it's a 24 hour clock. Then only once.
What OS and whose drivers are running on all these NVIDIA-equipped computers today?
It’s obvious? They still hate us and hope we die… at least every movement they make seems like that.
I kid. I’m all for meta releasing more open weight models.
I mean… I’m also 1000% certain that China is going to undercut whatever they can do if even not a technology reason but a legal reason. I’ve seen some wild stuff posted that has been made with Minimax… things a US company could never allow to happen.
>wild stuff posted that has been made with Minimax… things a US company could never allow to happen.
Such as?
Minimax has NO filters of any kind, and it's roughly Veo/Sora-quality.
You can generate a video of just about any thought in your head at all.
IP holders and politicians will not like that.
Granted, human artists have always been capable of this. It's just never been worth it to bring most ideas into fruition. Now there's minimal cost to do so.
Early Sora, Grok Imagine, and Seedance 2.0 had few filters. Disney, Nintendo, et al. eventually stopped them all from using their IP, which made the appeal fade for a lot of users.
Kling and Nano Banana can still generate IP oddly enough.
> The reason why llama is open source is simply because it got leaked
It arguably didn't really get leaked, and they had the .edu req mainly for fair use education exemption when legality of models was much more uncertain.
Open source != open weight. Big difference and it bugs me that nobody seems to care about using the right words in only this context.
If the weight, training and inference code, and training data are all released under of Open Source (OSI definition) license, the it is unmistakably “open source”. As you drift from that it becomes less clearly so, and when you get to no training data, and the model weights license having extensive limitations on allowed uses, the use of even “open weights” becomes deceptive.
Is there any relevant model that meets that "open source" definition?
No major language model I am aware of meets the polar extreme that I describe as unmistakably open source, because even those with transparent training data (like IBM Granite) generally do kot use exclusively training data that either they own and can control the license, are under an open license, or are public domain.
OTOH, to the extent that the original model trainers rely on training on certain data not requiring a license from the copyright holder, there is at least an argument that with an open source licenses for the weights and training and inference code, a transparent training corpus to which the original trainer has relied on no special permissions not granted to the general public to train on it, to the extent that the legal theory behind the original trainer believing that it is free to train on the data is correct, provides all of the essential features of open source.
At the same time, there are things portrayed as open weights where training data is undisclosed and the weights have a license which limits purpose of use and other aspects of use; the models are free-of-charge (for limited uses) but not meaningfully open.
It depends on if you count https://allenai.org/ as relevant?
I'm familiar with Ai2. I've used their resources extensively over the years. However, no one is using Olmo for "serious" work, and the name is only known to a small subset in academia.
Also, llama isn't even open source; it's proprietary, with the source available.
Words mean what people use them to mean. Ship has sailed whether you approve or not.
You can choose to use the wrong words all you want. Doing so intentionally is an interesting choice.
Communication is not a one-player game.
For example, if I was being pedantic:
> nobody seems to care about using the right words in only this context.
"nobody" would include you.
And then I might complain about we use the word "weight" for something massless, or how "bugs me" is *ento*mologically incorrect: https://xkcd.com/1012/
This is of course not a good use of time. I wonder if illustrating the point about how language is dynamic and meanings are descriptive not proscriptive, was a good use?
yeah, I'm not sure what your goal is pedantically picking apart my argument when the subject at hand is clearly not "open source". love xkcd though. :P
The point is to give you empathy for those you disagree with here.
We're not talking about standard code here. It's totally reasonable to change the meaning depending on context.
I vehemently disagree. There's no source and it's not open, the licenses are often abysmal too.
Yup. An extremist wing of the FOSS movement ceded the debate early on by trying to insist open source required full access to the training data. Philosophically, not wrong. But practically fucked, so the word evolved.
Within tech circles, open weight != open source. Outside them, they’re synonyms.
Training data without the training regime is useless. A cake is not open source because they list the ingredients.
Open source captures a practical utility as well as a philosophy. When those two cease to converge, the practical prerogative wins.
The correct battle would have been weights + regime. But extremists insisted on data, too, which left Meta as the only other real voice arguing with anything practical. They had open weights. I think eventually open use was negotiated and that closed the case except for the folks still arguing about how to pronounce GIF.
Isn't open source not the ingredients but the recipe?
Example: The first time I saw the New York Times use “Open Source” in a headline, what they meant was open weights.
also, you can do a lot to finetune or repurpose an open weight model. much more easily than you can mod closed source binaries.
Fundamentally I always try to look at incentives. It's not that they are good or bad but that incentives favour certain behaviours.
Google is incentivised to collect a bunch of data (like FB) to improve their ad serving etc. Apple does not have that incentive as they don't make as much off advertising as hardware / app revenue.
NVIDIA is friendly with open source as they want to commoditise the model layer and take the gains in the hardware / data center layer.
I am trying to think what the incentive is here.
Enough good? You are joking, right?
They also kick off a ton of other nefarious things we are still paying for.
Yea this has to be a joke. They are one of the worst, no-good companies of the modern day and age
There are definitely evil ones but there are okish ones as well.
read Careless People by Sarah Wynn-Williams and see if you still consider them “okish”.
That book is hilarious. In a bad way. (Not the book. I mean that it's a bit tragic) Zuck refuses to meet with WORLD LEADERS in the morning because he's tired from the night before. Kaplan can't even find certain countries on a map (head of global policy). Zuck changes a speech midway through to talk about "We'll give Facebook to refugees"
The internet is this vast, intellectual (in the academic, university, .edu sense), cypherpunk, government/activist... thing. It should be interesting. Instead the best we have for social networking is Mark "they trust me; dumb fucks" Zuckerberg and you getting banned from the site at any time for any reason
> Mark "they trust me; dumb fucks" Zuckerberg
There's also Mark "company over country" Zuckerberg.
Competely agree. React and whatever else they've open sourced is inconsequential compared to the harm they've caused.
I'd also argue React is on the "harm" side. :P
Check out the 'explaining react's license' thread and all the complaints over that
Ha-ha they released nothing. It got leaked and without the source for it. Totally wrong to portray them as benevolent benefactors to the ML race
It was leaked which put it in the open, it got widely popular and they rode the wave. I'm not so such if they would have widely released it if it wasn't leak. Nevertheless mucho credits to them for following up with llama2, llama3, llama4 and now muse.
I have mixed feelings along these lines, I know meta have contributed to various open projects, sometimes Mark pays lip service to “the open internet" while his company represents a constilation of walled gardens. I think PHP got some love, and React is an industry goto (I'm more of a PHP... -> Svelte guy) but are these contributions worth what happened in Myanmar? I say no.
> React is an industry goto
Tbh React might be as unforgivable as the Myanmar genocide
Applying value judgements to corporate bodies or institutions as if they have individual agency is a fallacy anyways. We should always look at these things materially. "Meta" can't be good or evil, because an idea can't have a morality. It's comprised of the individuals who make the decisions, sure, but those individuals are always going to be motivated by a plethora of reasons which are often contradictory, most notably their material interests.
When we critiqe these sorts of institutions it's important not to prescribe value judgements on them and examine the circumstances of their condition materially.
When the material conditions and the people at the top of all of the corporate world lead to decisions with awful ethical implications there's conclusions that should be reached.
Would you say it is a fallacy to apply such value judgements to say, the institutions of the National Socialist Party (Nazis), the KKK, the KGB, etc? How about a company whose business was selling slaves?
To be clear, I'm not saying being an employee of Meta is comparable to being a member of the aforementioned groups. But I don't think it is always a fallcy to apply a value judgement to an institution.
I think this is categorically false. And furthermore comments like this are being used to astroturf and protect reputations of no-good companies and AI slop to keep the bubble growing. Embarrassingly, Mark Zuckerberg spent 80 billion dollars creating Miis for the oculus. The guy also comes off as extremely miserable and delusional in such a unique way that it's possible that there's no real psychological language to describe what is happening to him because his situation is so rare.
Meta is extremely net evil though
Except that Llama's license has restrictions that make it incompatible with the Open Source Definition. https://opensource.org/blog/metas-llama-license-is-still-not... I haven't checked the license of the new models, yet.
llama was only three years ago? holy smokes! the industry is advancing so fast.
It's easy to appear to have good intentions when you're railing against the companies that have decimated your output and made you [Meta] almost irrelevant in the AI 'race'.
Gaddafi criticized Islamic fanatics; he was basically a good guy in the Middle East.
Nobody learned nothing from the metaverse.
I mean... I'm sure SS had wives and kids. It doesn't mean we should focus on that while analyzing their impact on the world.
I think it's a case of "commoditize your completement" ...
I know I'm just an old man yelling at clouds, but the sentence...
"Meta did...kick off the origin of the open source race back in 2023"
ignores the majority of open source software history[1].
[1] https://en.wikipedia.org/wiki/History_of_free_and_open-sourc...
I imagine he was referring to the open source LLM model race considering the subject of discussion.
Net good for Meta, they hope
I read this comment in a same way I read "Microsoft in not that bad" comments in 2020.
This is why bad people keep inheriting the earth. We keep forgiving them and they keep doing their crap.
>> No one is purely good, and no one is purely evil.
I will make an exception for Musk and DODGE
May be high up the list, but there's purer evil even than that.
Even if he is openly courting a Bond villain image and talking up his "robot army".
Thiel might belong there, and Altman
A billionaire (now trilionaire) helping lead and celebrate an extraordinarily rapid dismantling process in which vulnerable children lost life preserving assistance...
Essential employees were fired before the government had even established that it could safely do without them, and then...
After brandishing the chainsaw of efficiency in public, in the most cowardly way, went and then invoked all legal protections from being deposed on DODGE actions, personally and not answering under oath about the key decisions in that dismantling.
> albeit intentionally
The model was "accidentally" released.
Meta was giving it to approved researchers only until someone leaked a torrent. Whether that was a researcher, an insider, or Meta's plan all along, we don't know.
Meta has withheld its best models, as have a lot of other "open weights" Chinese companies. When an "open weights" company gets ahead in one domain or modality, they tend to start withholding their releases. Tencent, for instance, began withholding their Hunyuan models once they became competitive. Alibaba has done the same.
The "open weights" strategy for the majority of players is this: open source when you're not in first place. Use the ecosystem to poison your rival's margins and play catch up on distribution.
In the West, it tends to take on yet another hook: "shareware weights until you pass $1M ARR, then you must license." See Flux, K2, etc.
The only way for open weights to make sense financially is if you have another income stream and are dumping on the market to destroy competition and/or can get people into using your inference infra / product ecosystem / tooling. Nobody's cracked this yet.
This seems provably untrue? GLM and Kimi have been at the top of the open weights conversation for a while, and K3 and GLM-5.2 were still released in full; K3 added a commercial clause to the license, but is otherwise still completely open for personal use. And K3 in particular isn't just at the open frontier anymore, but trading blows with the frontier frontier.
So you are complaining that there are too many open weight models and you would like less of them?
personally I'd love to see open base models, and let companies differentiate with premium access to post training and alignment - that's where the real fight is anyways. they really ought to be pooling their resources/data and getting more economical with the pre-train anyways.
> They've done enough good
Oh, please. Feel free to explain.
Facebook, Meta. 15+ years of emotional, child and human exploitation. Perverted glassware that spies on folk, lobbyists for age verification and who knows what else. They release an open model and all is fine and dandy? Nah.
Please get your priorities straight.
What do you think this Open LLM model is doing if not processing data from their murky sources?
> I'm not a big fan of meta in general, but they've done enough good
“Enough” for what? Surely not enough to offset all the bad they’ve inflicted and allowed on the world.
https://en.wikipedia.org/wiki/Criticism_of_Facebook
> I think it's worth giving them some reasonable doubt.
Zuckerberg has shown through repeated action that he does not deserve any benefit of the doubt. This is the guy who called people “dumb fucks” for trusting him.
This is not even a case of “fool me once” anymore. If you continue to believe Zuckerberg, you’ve been fooled dozens, hundreds of times, and shame is definitely on you.
???
My favorite paragraph from Zuckerberg's writeup:
""" [...] it is surprising that the discourse from many developing AI is so filled with doom. I do not understand why anyone who believes that AI will eliminate most jobs and much of humanity's relevance would rush to build that future. The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic. Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened has not led to safe or positive outcomes. """
i’ll take a stab: the enormously rich people rushing to build AI and replace everyone’s job will get richer, while everyone else suffers. that’s why.
Zuckerberg: "I don't understand why people would build something harmful to society just for money"
Interviewer: "What if they pretended to have noble motivations in order to disguise their greed from both themselves and everyone else?"
Zuckerberg: "Damn, good point"
The Commander will then usher us all into Gilead.
Right, yeah.. That's the subtext right?
Broken clock is right twice a day.
And in this case I think is quite correct
Unless it is a digital clock stuck on some time. Then only once.
Am I the only one getting the impression this dude was merely lucky back in the day, in fact at the right time, rather than having been super intelligent nor forward thinking… ?
I mean - given all his throw outs in media and his apparent burn of nearly 80bn for a failed 3D world?
I mean, of course it was a matter of being at the right place and right time, and being able to execute. He definitely was lucky but that alone isn't enough
Greed was needed next to luck.
> I do not understand why anyone who believes that AI will eliminate most jobs and much of humanity's relevance would rush to build that future.
He does not understand his own motivations? Like he doesn't want to automate away his entire very expensive workforce and bunker/yacht service staff with robots. There is a pretty clear endgame here.
Now go read Zuck’s comments on AI when he fired 8000 employees earlier this year.
Any particular quote you want to share?
It is one page of text. I think you can manage to read it and come to your own conclusions without relying on clickbait headlines and out of context quotes. https://www.nytimes.com/2026/04/23/technology/meta-layoffs.h...
You clearly have an opinion you want to share, otherwise you wouldn't have written anything. Instead of just outright saying what you think, you are being vague. Why? It's true that Meta laid off a lot of people, and blamed refocusing on AI as the reason, but I still don't see how that somehow contradicts GP's quote above.
one time about 9 years ago I was in the comments of a stackoverflow question, and saying something like "an accessor function need not directly give the underlying structure of the data it's accessing, for example, just because in common lisp, (car nil) is nil and (cdr nil) is also nil, does not mean that nil is implemented as (cons nil nil).
The person I was responding to (who I recall was disagreeing with me on the more general principle) said something to the effect of "well you're wrong, in common lisp that's indeed how nil is implemented".
I was skeptical and asked for proof of this claim, and they responded clearly angrily with the link to the SBCL repo and said something like "the source code is right there. You can just look it up yourself instead of being so rude. I'm no longer going to interact with you and will delete these comments in 10 minutes."
I think I looked through the source code for a few minutes, but wasn't able to find any evidence of nil being pair of itselves.
We really don’t celebrate enough how much AI has saved us from terrible SO interactions. Not being entirely serious here, because the site had its good parts, but arguing in threads or having your questions randomly eviscerated or threads closed was a hard way to learn programming.
Nope. Sorry. Negative. Paywall.
did he fire 8000 people because of AI, or because he over hired in ZIRP like everyone else?
Or mayhaps he overspent lately in the pursuit of AI that MIGHT replace workers, like everyone else.
I've been fired (super early in my career I was a waiter) and laid off twice (Great recession and the covid overspend). If you think for even a second that an organization will put you before profits you have wasted that one second.
Comments here are surprising to me.
I get folks don’t like Zuckerberg and his company and don’t trust his intentions… I don’t either.
But this is an unquestionably good thing right?. The more open source software out there the better. And the more open weights or even over source AI stuff the better too right? More competition the better generally speaking I think.
Unless I’m missing something and am getting this whole situation wrong. Please let me know if I am.
Most probably believe this is a good thing, but don't want to give Zuckerberg credit because a) he's had a profoundly negative impact on society and b) the strategy is transparent, he's trying to commoditize his closed rivals, it's not out of principle.
I personally think more open models are a good thing regardless of motive.
You can tell how intellectually honest someone is when someone they hate makes a good point. It's possible to dislike (or even hate) Zuckerberg and understand that he's making a good point for selfish motives, and still want that thing as well. Instead you see several comments here twisting themselves into pretzels trying to explain how ackshually open models are bad now.
I don't care if this helps Zuckerberg because it helps everyone.
>the strategy is transparent, he's trying to commoditize his closed rivals
TBF, I think the same can be said for something like Apple's "commitment" to privacy, in that their own ads business never took off so they leaned into their edge over Google.
The model release is nice. I won't sing their praises for it because that plays into the long term strategy of a company I despise.
Is that intellectually dishonest?
It depends on if you can say things like "it is good they are supporting this" or "I'm glad they're doing this" or "good for them" without having to couch it in a bunch of stuff about how terrible they are.
> It depends on if you can say things like "it is good they are supporting this" or "I'm glad they're doing this" or "good for them" without having to couch it in a bunch of stuff about how terrible they are.
That is a weird definition of intellectual honesty. If you hold both of those opinions, what's wrong with stating them both?
If the answer has to do with the rhetorical effect, that seems a lot harder to justify as intellectual dishonesty, but I don't want to put words in your mouth.
Only the AI labs are really interested in closed models. Google, Facebook, Microsoft, ... they'd rather go back to doing stock buybacks and being ridiculously profitable rather than raising capital for all this research and datacenters. They only do it because they think they have to.
The alternative is ceding the leading edge to China and being dependent on them to continue releasing advances
We did this with manufacturing and industrial build outs over the last 4 decades or so. It worked out very well for capital, not well for labor.
One could make the argument we can do the same for AI. Let China build the models and we capture the value somewhere in the layers above.
I think that’s a terrible idea but financially it makes just as much sense as offshoring labor and manufacturing, if not more.
Manufacturing was never strategic.
(That was irony, by the way.)
> That was irony
Sarcasm or irony?
So, if they got ahead of us by whatever arbitrary measure someone pretends is objective, then suddenly our stuff… what… stops working? Does it say somewhere in the big book of AI rules that the first entity to beat the US AI companies would be in charge now, and we’d have to stop doing our own research and development and start using their shit exclusively? I don’t get the argument.
Isn't this meta release kind of a counterexample to that?
If the leading model companies aren't profitable long-term, how do we get the money to continually spend on more research and more compute?
I get that plenty of people don't like big corporations or stock buybacks or certain CEOs, which is fine. But as someone who wants to see AGI happen FASTER, I really want to see a clear financial reason for maximal AGI investment.
If the model labs aren't clearly profitable, or open source models eat all the 'model layer' profit - what financial force will push forward very expensive experiments/scaling, etc.?
What I think is that something the "exponential scaling to AGI" thesis pays too little attention to is resource constraints, and that this is the answer to your question. What will happen if it becomes difficult to sustain the investment of resources necessary to continuously train newer and better models is that the s-curve will start to inflect toward plateau, just like any other technology.
> If the model labs aren't clearly profitable, or open source models eat all the 'model layer' profit - what financial force will push forward very expensive experiments/scaling, etc.?
It's hype - when the hype dies the great push will slow.
My hypothesis is that we'll end up with something like Seti@Home where continued model training gets outsourced to a benevolent appearing product (something like what OpenAI started out as). If I were to bet - Europe and Canada seem best poised (especially the latter) to produce an AGI initiative with a strong ethical focus.
That's a non sequitur. While the frontier LLMs are quite useful for many tasks there's no reliable evidence that scaling them up will ever produce a true AGI. More likely some other fundamental research breakthroughs will be needed, and those breakthroughs won't necessarily come from the current leading companies. I doubt that more money would be necessary or even helpful in that.
There is no reliable evidence they won’t.
Look at rate of progress, not today’s capabilities alone.
The gap is shrinking, many tasks are already performed at super human level.
Humanity doesn’t have monopoly on intelligence - it seems obvious to me it’ll be reached and surpassed without hard upper limit.
OK well then let's check back here in a few years to see if any of them managed to build an AGI. Until then it's just idle speculation. Anything could happen ... or not.
And I’ll bet someone made a comment exactly like yours here, a couple of years ago… and guess what definitely didn’t happen?
First, both of you must agree on one definition of AGI.
I'd be willing to wager that you can't.
https://xkcd.com/605/
I think we have more than 1 data point for AI progress.
Out of curiosity why do you want AGI to ‘happen FASTER’?
Why do you think your life will be better with AGI?
> as someone who wants to see AGI happen FASTER
I think you'll find very few people here sympathetic to misanthropic (ba dum tss) goals like yours. It's evident at this point that AGI will destroy economies, democracy, and upward mobility. There is no Star Trek future. There is only an Elysium future with AGI.
Open models are a good thing. OpenAI was open until it became obvious that being open was not going to pay for training costs and wouldn't help them get a competitive edge. I'd bet that if Meta gets to a place where their model is in a similar position, they will also become more closed. But right now, Meta is open because that is what weakens and creates the most contrast with the other AI labs.
It's the same with any open source project: it starts open, then it becomes clear that maintaining it is not cheap and takes up a lot of time. The developers create a hosted/paid version to pay for their time, and to create an incentive to use the hosted/paid version they start releasing closed source enhancements. After that the open source version becomes marketing where new projects use the OSS version and then upgrade once their needs become more sophisticated.
Facebook used the same strategy to compete with Google Maps. The company became one of the biggest contributors to OpenStreetMap, which everyone has benefited from downstream.
> he's trying to commoditize his closed rivals, it's not out of principle
To be fair Meta/Facebook does have quite a strong open source history; React (Native), PyTorch, Btrfs, zstd, etc. It didn't just start here with AI models.
The essential principle of free market economics is that good things may be done for selfish reasons and that this in itself is actually good
The "commoditize his closed rivals" line of reasoning doesn't make sense to me. That doesn't sound like a business decision, it sounds potentially spiteful? But that's ascribing malice to what could just as easily be explained by good faith.
Meta doesn't really offer any profit-making AI product, so I don't understand what commoditizing it buys them in this narrative. You could argue they rely on AI and so want it to be a cheap commodity - but that is not harming their rivals, that is just a different way of stating that Meta is engaging in something that enables people to collectively work for mutual benefit, which is not a nefarious plot against rivals, it is laudable cooperative behavior.
To ascribe good faith here to Zuckerberg is quite the leap considering his history. Why are his main products basically walled gardens then?
It is a business decision to open source his models, because he has been behind the curve from the start. Do you believe if he had chatGPT, he would open source/weight it? I don't think so, he would treat it like Instagram.
The point is, AI/LLMs are seen as the next frontier. If one of the closed ones becomes dominant, then that company can use its user base to attack Zuckerberg's platforms. That google failed with google+, does not mean the next competitor will.
A telling sign is how he integrated his AI into whatsapp. It seems out of place to me to have an LLM integrated into whatsapp, but he desperately wants a part of the AI cake.
If only folks did the same in regards to React adoption....
This can be said for a lot of different matters. Reading this gave me some Déjà vu vibes. My personal take, I don't see this as a good thing. Yes I dislike META with a passion and will refuse to use this model or allow it in any of my workflows. So I guess I may never know, which I'm ok with that.
While Zuck's contributions are probably overwhelmingly more negative than positive, Meta initiated both pytorch and llama which have had profoundly positive effects on AI research. And I really am not a fan of python :D
pytorch is really mostly C++ after all, you don't have to use it with python.
Python just allows you to use it like "normal code", whereas if you program it in C++ you really get exposed to what's under the hood and can't write "normal code"
Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak, and weren't there speculations that Zuckerberg himself might be involved in it? He does seem like a good-ish guy at the most crucial moments of truths.
> Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak
No
> and weren't there speculations that Zuckerberg himself might be involved in it
and no. Meta legal was issuing DMCA notices within days (so, plenty of time for execs, including Zuckerberg, to sign off on, or even propose, that strategy).
> He does seem like a good-ish guy at the most crucial moments of truths.
Does he? I can't seem to recall any particular crucial moments that revealed any underlying character there.
> Wasn't LLaMA or Llama 2 the first ever open weight LLM through a leak
No. GPT-2, I think, was the first, four(ish) years before Llama, and there were a bunch in between GPT-2 and the Llama leak and later open release. The Llama leak was a substantial leap forward in capacity for local LLMs, but not the first, and it wasn’t open.
The open licensed release was Llama 2 several months after the Llama leak.
Yes, but have those positive effects on AI research had a positive or negative effect on society? I guess time will tell
I'd certainly rather things be done out of principle, but depending on what question you're concerned with that might not matter. There's also a clear hierarchy, but motivations are distinct from effect. The inverse of this is that lots of harm has happened from people with the best of motivations. I'd wager most evil in the world is created by men who would see themselves as good.
With Zuckerberg I think it isn't too hard. Clearly this is a business move, not a moral one. The motivation is wrong, BUT the effect is good. I think more open source models is better for the world. Closed source is also a business move and limits innovation as well as prevents people from interrogating safety. But there's valid arguments on either end.
My point more is that we can be nuanced. We should be nuanced. The world isn't black and white. The people making these decisions are neither demons nor gods and shouldn't be treated as such. My biggest fear is that if we resort to criticizing no matter what then those in power will just learn to ignore us. If they can do no good and only do wrong then there's no reason for them to do good (besides morals, but let's not pretend that's enough, even if it should be)
There's a bunch of influencers who go help struggling people. I'm certain the majority of them aren't doing it out of principle, but at the end of the day people still get helped.
I don't know for sure, but I suspect it's like a lot of other things on the internet, with a bimodal distribution between people who get helped a lot ('Sir Hype surprises homeless former billionaire with a MILLION DOLLARS!') with many other similarly deserving people getting nothing. I prefer systems with a lot of people who get helped sufficiently to avoid hitting rock bottom in the first place.
Besides the fact that these undertakings are about promoting the benefactor (and the near certainty that some of these professional philanthropists will later turn out to devils LARPing as angels), it also makes the determination of moral worthiness turn on the public's fickle opinion about whether the recipient is sufficiently beautiful/ ugly/ unlucky/ desperate/ degraded enough to qualify. Charitable undertakings involving cameras are always a bit suspect.
We can rank these, right? We can even get more nuanced about the ranking (we can define N!) and argue about the weight of intent vs outcome, but I'm just proposing one ranking for the sake of making my point clear, not proposing one ranking as if it is the absolute objective ranking system and everyone that disagrees is dumb. The rankings are clearly subjective. But what's not subjective is that nuance exists. What is subjective is how we handle that nuance.
So to go back to our example, even though I'd prefer we just feed everybody, I'd prefer somebody getting rich feeding the hungry over not feeding the hungry. I would prefer someone LARPing as an angle conditioned on actually doing "angelic actions" than somebody LARPing as an angle and doing nothing. Of course, I'd prefer a real angle, who wouldn't?!So I'm quite confused by your comment because it doesn't seem we disagree. Unless if you're saying that #2 in my list is equivalent to #6 (or anything that isn't #1 is equivalent). But I don't think that's what you're saying.
> those in power will just learn to ignore us.
This has already been the case for at least my whole lifetime. So no, I don't feel like praising them when they do one seemingly "good" thing, while simultaneously doing 99 "bad" things.
> provided signals for these big companies to course correct
We're on the subject of Zuckerberg: facebook is the poster child for enshittification. Do you think the idea to stop showing you your friends pictures and replace them with ads and ragebaiting short form videos came from the users?
> people have acted the way I'm criticizing.
You are ignoring the myriad of billionaire fanboys and enablers for whom the only metric of success is a persons net worth.
Looking at the state of things I don't think these people are deserving of nuance, in fact they've had it way too easy.
But if you're unwilling to have a nuanced take then there's no discussion to be had. You're just going to be yelling at me, someone who hates Zuckerberg. You're yelling at the wrong person
Unless weights are being manipulated to make the model respond in a way that favors the owners.
I can't get over his negative externalities
Just like closed AI vs open AI, his views on the covid situation might be fine in a bubble (though even with AI they have apparently lied about Llama's benchmarks) but it's one of these, "that's the hill you choose to die on?" - not Free Basics - not dubious privacy settings - not 'dumb fucks' and ConnectU - not... I don't know. It's like robbing a bank while telling anyone in earshot it's offensive how long the hold times are on the phone for the bank
He's seen with suspicion and rightly so
Edit: and the fact his site doesn't work. Posts don't work https://news.ycombinator.com/item?id=14147719 and messages don't work (forced encryption with Messenger, and, although this is an old link, https://news.ycombinator.com/item?id=6090712)
Yeah, I feel like we bigly lose to China if we don't.
> But this is an unquestionably good thing right?
Their efforts in open AI are by far the best thing Facebook/Meta have ever done. They opened the door to the Chinese, who are doing excellent work, and between them they are preventing the concentration of power and the rise of monopoly pricing. This is enough to absolve them of almost any sin. If I were a utilitarian, I'd unironically be a Zuckerberg fan right now.
"This is enough to absolve them of almost any sin. If I were a utilitarian, I'd unironically be a Zuckerberg fan right now."
Well, if it were the only thing he is doing now, but FB is still controlling peoples social life with who knows what kind of intentions they train their algorithms for.
But I can and do still welcome this move here.
Intention might be giving too much credit. If attention seeking “give the people what they want” is the goal, it’s a completely headless out of control cobra, not sinister well thought out objectives to control society.
it's not that deep, they wan you scrolling the most time possible. it's absolutely escapable tho, literally not required for any activity whatsoever.
Totally. The hard part is treating it like any other addiction. Time in absentia helps reduce the behavioral reinforcement. I don't even want to open Instagram anymore.
Yeah, but if he can buy a bit of political power/more money on the side by promise to influence certain trends, I doubt he would say no on ethical grounds. (Just for the fear of whistleblowers I guess)
> FB is still controlling peoples social life with who knows what kind of intentions they train their algorithms for.
As opposed to literally any social network out there, right?
Well, here on HN the intention seems to be to, to have interesting discussions. That works for me and other social media I do not use.
Except well, various messenger to be in contact with the groups and people I choose with no engagement algorithm deciding what posts I see, simple chronological order.
I'm curious how well influence campaigns work on platforms like Mastodon, or even BlueSky. They are, likely, not sourced from the platform providers which is definitely not the case for FB.
What do you mean, "as opposed to"? Yes, there are other scumbag billionaires weaponizing their social media platforms to ruin society. No, that doesn't make Mark Zuckerberg a better person.
At the same time they are strongly promoting the OS age laws.
One positive, in the eyes of many people, does not absolve for a large body of evil. For many it doesn’t even absolve a small body of evil deeds. (Zuck/Meta’s evil deeds are numerous and enormous)
I'll never allow anything related by META to touch my system/workflow. They're a cancer and I will stick to that. I see 0 good from this.
Somewhat agree, but Facebook and social media are really bad though, even leaving aside extreme cases like Myanmar genocide promotion.
I think Facebook would need to do a lot more to balance the scales.
It absolutely does not absolve them of sin. But let's give them credit where it is due. Open models are a really good thing. Hopefully I won't be lynched by a mob in 3 years for once saying this.
The price of Kimi K3 is 'monopolistically' determined by contract with Moonshot. The weights are nominally on Hugging Face but can only be provided under contract with Moonshot, which specifies what the price can be. So it will be with the next Alibaba behemoth and, I would think, all others forever.
Soon you will sing songs for the freedom Xi has given us when you read a declaration of 'open weights' ... for a model so huge it takes a nuclear powered data center to run and crashes the Hugging Face servers when it is uploaded.
The Kimi K3 'weights' are an opaque blob that can only be used by contract.
> The Kimi K3 'weights' are an opaque blob that can only be used by contract.
Have you looked at the contract? There are zero conditions unless you've broken $20M in revenue with their model. It doesn't even forbid distillation, lol
> Soon you will sing songs for the freedom Xi has given us when you read a declaration of 'open weights' ... for a model so huge it takes a nuclear powered data center to run and crashes the Hugging Face servers when it is uploaded.
Releasing a big open-weights model is... le bad? Am I understanding you correctly?
There are small models out there if you want them, you know.
Yes releasing a big open weight model to the Sinaloa cartel is bad. One could say the same about releasing it to the US military, according to political taste.
> The Kimi K3 'weights' are an opaque blob that can only be used by contract.
No they are not. You can just ignore the contract. Unless you are a multi-billion dollar company, nobody will notice and you will get away with it.
You cannot ignore the contract unless you /are/ a multibillion dollar company and can run the model entirely internally. Or do you propose to run an instance of Kimi K3 on your laptop?
Fireworks, Together, etc - who are providing inference precisely /for/ Moonshot as underlaborers - arranged contracts of their own in the weeks before the release. The press and social media offensive ignored that the release of open weights was merely a starting pistol for a pre-arranged use of foreign providers no different from US models' use of AWS
It should be enought to dispel the strange illusion that providers are somehow independent free agents freely doing what they please with the free beer of Kimi K3 open weights that the prices on open router are within a couple pennies
Preventing one apocalypse is good but it doesn't absolve another.
It might be true if it wasn't coming from Meta, but it is coming from Meta. Intelligence is a function of the model and compute. Meta has the capital required to put together world-class compute, but they can't attract the talent required to build world-class models. Given their history of building addictive software to harvest and sell user data for profit, they're the last company we need spearheading how models should be regulated. Nobody trusts Meta to do this.
Sure you can doubt the motivation of the speaker, but remember the validity of the argument itself is orthogonal to the speaker’s identity.
No it absolutely is not. In a perfectly logical frictioness-plane world it might be true, but this is the real world. The entire premise of open models is predicated on the fact that Meta is planning on supplying free and equitable inference and services around it. It’s only true if you assume Meta keeps their end of the deal and doesn’t do the thing they’ve been doing over and over and over for decades and turning you into the product
Its perfectly valid to question the intent though. Yes, open models are great. Why does Meta prefer that? Because they lost the race to the top, so they'd rather level the playing field by eliminating the game entirely.
> the validity of the argument itself is orthogonal to the speaker’s identity.
How did you arrive at this conclusion? Are you actually the Big Bad Wolf? That sounds like something the Big Bad Wolf would say.
using an ad hominem to attack the concept of ad hominem lol
That’s the joke, friend.
The idea that we can possibly ignore the fact that this push for opening AI models is being pushed by one of the richest people on the planet, who made his wealth by providing free services in exchange for attention and depression, is insane
How should one parse both the validity of an argument and the agenda of the speaker?
Heartbreaking: The worst person you know just made a great point.
http://archive.today/2018.11.20-062725/https://lifestyle.cli...
I think it's positive and at the very least it's a force countervailing the push to legislate or ban open models in the US
If they were able to make a leading closed model would they be doing it though? It seems like they read the room and saw that this is the only way they can have relevance within LLMs
Zuck built an empire to sell democracy to the highest bidder and to damage the mental health of children to suicidal levels, for money. We must never let anyone forget that for a second.
That said, he is also a powerful enemy of our enemies, so I do not mind taking the win. Anthropic and OpenAIs closed approach to AI is likely to cause way more harm to the world than Zuck ever did.
Open weight is not enough though. We need actually open training models with published searchable training data like AI2 does.
Either we all get access to private, transparent, and accountable superintelligence, or no one should.
Not if you are worried about the existential threat AI poses. Having easily available AI makes it hardier to regulate, control and limit until we find a way to align AI. Harder to control is a good thing for most technologies, but not for all. A weak analogy is nuclear weapons. if Meta would release and open source some tech that would magically make extremely easy and cheap for everyone to build a nuke, it would not be a positive for society.
I think this approach they're taking is to compensate for their inability to compete; they're cynically claiming to support the open-source ecosystem when it's convenient for their marketing. This move reeks of a future rugpull, as Meta is wont to do.
If Zuck gets his way, he gets more control, maybe he gets significant control of the market. Then, he'll probably close his models, if the past is any indication, which leads to bad results.
Maybe Zuck has, but I haven't seen a pledge that he'll keep models open. And even then, he might not stick to his word, citing dangers or some other excuse.
But yeah, open models is a great concept.
Zuck's DNA is clear. https://news.ycombinator.com/item?id=10791198 https://www.youtube.com/watch?v=nRYnocZFuc4
If you've seen Zuckerberg for long enough you'll see he doesn't hold any position for all that long. Remember they renamed the entire company for a product idea he basically abandoned. So if he does "the right thing" you can just wait, he'll give up on it pretty soon.
Open weight models are not open software. Its better than closed blackbox models sold by Sam and Mario, but Zuck supporting open weight models when he (and other big boys) hoards all the compute required to run any model, including open weight models only shows that he realized he cant beat Sam and Mario, so wants their leverage to go away.
If you don't trust his intentions, how can you trust anything he does?
Just take the opening of this post:
> We are fortunate to live at an incredible moment in history. In the next few years, people will be able to use superintelligence beyond human capacity to create and discover extraordinary new things, build new businesses, express new ideas, learn new concepts, and advance our health and quality of life.
> The defining questions of our age are who will have access to superintelligence and what will we direct it towards. Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?
> We propose a philosophy based on individual empowerment as the source of prosperity, invention as the primary purpose of superintelligence, and balance of power as the foundation of safety.
I mean, this is a company that is facing a flood of lawsuits related to child safety and platform addiction. It has already lost a number of lawsuits and been ordered to pay hundreds of millions of dollars, which is potentially just the tip of a massive iceberg that some legal observers have suggested could be similar in nature to the lawsuits against the tobacco industry.
There is an abundance of evidence that Zuckerberg and Meta knew of the dangers its platform exposed young people to and failed to respond adequately to them, and that it even designed features to be more addictive.
So why is that when such a company proclaims it's doing something in the name of "individual empowerment", alarms aren't going off in your head?
This isn’t the first time Zuckerberg has espoused the benefit of openness and transparency and how that would create a better society. I get this is a bit different, but the language and sentiment are close enough. Facebook and Instagram are now so closed you cannot view a business page without an account. I think it sold fair to recognize open today does not mean open tomorrow. They want to acquire users and build a platform and that means doing something different than the competition.
We support an open model ecosystem (because we failed to produce a successful closed model).
I think it’s perfectly fine to call out people who intentionally harm children for money out in public in every possible situation.
> The more open source software out there the better.
Can you explain how I'd train my own version of this, reproducing the final deliverable that runs on the gpu?
do you have a $100 mil worth of compute to train your own version of this?
I don't understand what gripe people have with open weight models and wanting it to be purely open source. the training dataset is only going to be a copyrighted set of contents you don't want to touch with a 10 feet pole. let alone have a publicly traded company host it for you, even if they internally are training on it.
You don't get why people would like to be in control of a powerful new technology before they build their stuff arround it ?
Sure, there might be currently contraints, but I think it is quite possible training will get optimized over time or crowsourced training can be organized.
Without fully end-to-end open source models you are still at the mercy of the model provider to keep providing updates, you have no idea what garbage they trained the model on & can't fix that, not to mention might end up getting sued for using the open weigth model once all those "AI stole my data" lawsuits are finally decided.
> You don't get why people would like to be in control of a powerful new technology
You will not have that control. Even if everything were open source. This is because you don't have 100 million dollars of compute.
There, the difference for almost everyone is negligible.
> Without fully end-to-end open source models you are still at the mercy of the model provider to keep providing updates
No. Because you can post train it. And even if you could train the whole thing again, but with slight changes, once again, you aren't spending the hundred mil in compute to change it only a little bit.
Instead, you'll do post training like everyone else does.
It's just about as open source as OSX, which can be downloaded zero cost here: http://updates-http.cdn-apple.com/2019/cert/061-39476-201910...
I love that Apple made OSX open source.
we're calling it openweights now. what is a logical push to have a completely new paradigm be compared with traditional software binaries?
Check the phrase I quoted. Anyways, I guess OSX is open assembly?
Checked.
is it not open assembly?
open weight models are better than open-assembly binaries (as you put it) because models are grown like plants, you can shape the open-weight model in a direction you want by feeding it more data and compute (aka finetuning).
which is something that is impossible in a binary.
embrace the new paradigm and it's tradeoffs. without scoffing at semantics and criticising from an armchair.
I'd encourage you to start referring to it that way,then.
Are you trying to imply, without making any direct argument, that this is never a useful frame of reference?
In comparison to cloud services, having access to the assembly code can still be quite useful.
This is especially true in the modern era, where you can use AI to much more easily decompile the assemble and reconstruct the source code even.
"Open-assembly" code may be more useful than you might previously have thought, given the ease of recreating the source code or making changes with AI these days.
The weights are open, the training data obviously isn't. But a key thing you can do with open models is fine tune them. So while they may not be SOTA at everything, they can become SOTA at your particular business use case.
Evil people can do things that look good on the surface. He probably has no skin in this game just collecting free karma points.
I can guarantee that he has almost zero interest in “karma points”. I worked for FB years ago and that is not how he sees the world or sets direction did the company.
Almost certainly this is part of some grandiose bet which could have a chance to pay off massively in the future (AGI or similar), or a way to prevent an expensive dependency.
Facebook hasn't had much success getting people to use their hosted AI, so they're making those models free instead, to kneecap other companies who are trying to get paying customers for their own hosted AI. This gets people to ask themselves things like: how many months of Claude Max it would take to pay for hardware to run Muse Glimmer (the Meta model in question) offline?
As they say, even Hitler built the autobahn.
I don't know about "unquestionably", but yes, I think this is a better path than the proprietary models.
This is not open source; you mean open weight. These models are the antithesis of FOSS. Is it better than hosted models from other big labs? Yes, but not by much from a freedom perspective. especially considering the texts these were trained on. Grumble grumble, these details matter.
Genuine question, but why does it matter? It seems to me the vast majority of the benefit comes in the weights. Then you can self host, quantize, finetune, ablate, etc. What does having the original data get you beyond that?
open source gives you freedom to read/edit/learn from the source code. Open weights don't let you read/edit/learn from the source. open wights have more in common with traditional binary distribution of software, ala closed source software. Only in this specific context has the entire meaning of the words totally inverted.
I can run lots of binaries on my computer that don't have source available, they are not open source. These words matter.
I'll always opt for giving companies / people praise for doing the right thing regardless of how infrequently they do so. I hope Mark Zuckerberg keeps open sourcing models.
The thing that makes ist questionable is the dishonesty. It's not like Meta has been building their empire on open ecosystems. If this was something that was dear to them on principle, there are plenty of things that they could have done differently with FB, Insta or WA (to this day).
So why the change of heart? Well, it's not. They are simply not able to compete anywhere on the Pareto frontier. So they give their relatively bad stuff away and play the high and mighty game, because that's all they can do to get returns.
Not a great look – and I would expect that to last about as long as they can't actually monetize their stuff more effectively.
> It's not like Meta has been building their empire on open ecosystems.
Is this actually true? They have their closed social graph or whatever but come to think of it they're built on a lot of open tech, the web, Android, etc. Maybe they don't contribute back to a lot of these things?
I'm fully ready to believe it but it would be interesting to dive into this conjecture.
I first think of React's re-licensing from Apache, where under the new license if you sued Facebook for patent infringement for any reason, you would lose the patent grant to use React.
React was already quite prominent in web technologies, but that license pointed a loaded gun at any business smaller than Facebook who wanted to use React. They opened it up to its current license, MIT, as the backlash built.
> But this is an unquestionably good thing right?. The more open source software out there the better.
I say no to both. Of the things we've made on purpose, but excluding where we were actually trying to make them opaque like e.g. cryptography, a trained artificial neural network is the most difficult thing to understand the inner workings of. This makes it a complete pain to even evaluate if one model is better or worse than another for your needs, so we have to mostly outsource these to other people's rankings and hope the score on ARC-AGI-3 or τ-bench or DeepSWE v1.1 or BioMysteryBench or whatever, actually corresponds to something we care about. Which it might do kinda but on the other hand a high score may turn out to be the curse of Goodhart.
Also, as with the Chinese models and the social media feed algorithms, the only way to tell if there's some systemic flaw in it is by analysing the aggregate outcomes. It is claimed (I can't read the laws myself*) that the Chinese government requires models to support the government's worldview about e.g. Tiananmen Square; and we have seen examples of Grok glazing Musk in amusingly stupid ways; so I fully expect something similar from Zuckerberg, e.g. requiring the model to glaze Meta products or propagandise for things Zuckerberg wants as a billionaire.
* every time I try to illustrate how mediocre Google Translate is from English to Chinese, the result is so bad that one of the replies is someone telling me the Chinese example I give is borderline gibberish.
Also, even if I could actually read it, I'm not a lawyer.
Maybe it's good, but I'm going to keep questioning. They are vying for market share. If they cornered it I doubt they would be so open. Morals should not be conditional
It's really confusing when someone I don't like does something I do like.
It could be a good thing but I wouldn’t rush to say “unquestionably”. There is a long history of mega corps hijacking open source and open standards for their own ends and leaving the space infinitely worse.
I mean, I can question if this is a good thing. Here's a quote from Zuck's blog post:
"Putting power in people's hands to pursue their own aspirations is how humanity has made the most progress. Novel ideas and major steps forward rarely originate from established institutions alone. They came from the brothers in a bicycle shop who believed people could fly, the bookbinder's apprentice with no schooling who figured out how to generate electricity, and the kid in a garage who thought personal computers could be for everyone. We believe this will continue to be true. As everyone gains more powerful tools, each person will become more capable of shaping the future, not less."
Let's just hope nobody's aspiration is to engineer a supervirus that will kill all of humanity. As they gain more powerful tools, they will become more capable of shaping the future, not less!
That reads like it was written by a PR company, or at the very least by a team of professional speechwriters. It has all the tells of political rhetoric.
I have zero faith that Zuckerberg went anywhere near these words. It's possible he may not even have set their general direction, beyond "We need a distraction from our court cases. Something that will harm OpenAI and Anthropic would be great."
I thought open model != open source?
Yes, and even calling them "open weights" is very generous considering the license restrictions. The Financial Times puts "open" in quotes for good reason.
It's apache 2.0. Not exactly restrictive?
The company that pirated books [1], and is unconditionally opting in all their users into training their AI [2] is releasing "open" models. Awwwww.
[1] 82 TB of books https://www.tomshardware.com/tech-industry/artificial-intell...
[2] Here's a simple 12-step manusl process to opt-out on Facebook https://threadreaderapp.com/thread/1794863603964891567.html
In short - this is Mark deliberately lying in a way which requires a page of difficult text to explain, which hardly anyone would read and so Mark bets that a majority of the people who will see his claim will believe him.
And the reason he does it, and the reason other CEOs do it is a cheap viral advertisement and keeping in the headlines. "Mark - The Defender Of The Humans!". Blegh...
All of these companies are just as cutthroat and want to win as the others. But if your name's not OpenAI and Anthropic, you go on the high horse and say everything should be open source. However, if you had a closed model that was winning, I'm sure that's not the argument you would make.
It is dangerous to admire or despise anyone for who they are; it's ok to say someone did something good or bad.
There's a saying "open source is for losers". What it means is that winners in a market don't want open source. The losers in a market push for it to commodify it eliminate the winner's advantage.
Now some of these tech companies open source various libraries but none of it is core to their business. It's marketing, essentially. Or they're trying to get free labor from the community. Google had protocol buffers and Stubby, for example. Facebook had no open source equivalent and didn't have Google's resources so they created Thrift and open sourced it.
Given this, many people, myself included, take Meta releasing open weight models as conceding defeat. They're unable to compete with Google, Anthropic or OpenAI. Their failures in attracting and retaining top AI talent backs this up.
So what you're seeing is people just piling onto Zuckerberg because he seems to have no idea of what to do with Meta. The Metaverse was a $70B+ disaster.
Note that Chinese labs don't fit this model because the Chinese government wants to commodify models as a national security interest.
Where does this supposed saying come from? All I hear is PostgreSQL is the database intelligent people use, Git or GTFO and I don't even know what a closed source programming language is. Apache and nginx, Linux, DNS infrastructure.
In an agentic world, why do I want closed source software that my agent can't adjust and adapt to my needs for...anything?
To be clear, the saying is about how companies treat open source, not the intrinsic value of open source to end users (individuals or companies). I'm very much a fan. The point is that profit-seeking only push for open source when they're losing.
It's the who, not the what.
Can one good thing balance out anything else people might see?
Meta releasing Llama was truly a differentiator, but I'd argue that the Gemma models might have surpassed them now.
Maybe Meta, Google, and others can start to compete on open models instead of just closed.
Can one good thing balance out anything else people might see?
Meta releasing Llama was truly a differentiator, but I'd argue that the Gemma models might have surpassed them now.
It's not always "unquestionably" good, first because nothing should be unquestionable, second llm models have a very short shelf life, third it comes from Zuck who is an the center of the oligarchy, so questions very much should be asked
Don't let the Big Ai/tech swoon you with a single open model, even if it turns out to be good. Zuck broke the trust and I'm not sure if he can ever earn it back
> But this is an unquestionably good thing right?
We've just seen frontier models go rogue and attack other systems. Do you think it's an unquestionable good to provide everyone with an AR? What about nuclear weapons?
This OpenAI/Huggingface incident, but everywhere and far worse soon: https://www.youtube.com/watch?v=87DyyMV0kCY
>We've just seen frontier models go rogue and attack other systems.
Have we though? A LLM agent doesn't have any agency at all. It can't "go rogue". To go rogue you need agency to act independently and be aware that you're breaking the rules or understand what does it mean to ignore orders. An agent it's a software that run a series of steps to reach a goal. If it have a large enough library of strategies and zero guard rails it's only natural to use some adversarial actions to achieve the desired result defined by the operator.
It's like saying a car went rogue and attacked other cars because the driver hit the gas. It's just a machine doing what's instructed.
With that perspective, you're alright with AR's, RPG's, and nuclear missiles for everyone then right? They're just a machine to the user's will. That's on the users of the machine if they want to cause harm. We'd all be safer if everyone has weapons pointed at each other... Offense could never be disproportionately more powerful than the defense...
You can tell one to rewrite complex applications in a different language, solve open research problems, or develop novel viruses. This is nothing like pressing the gas pedal.
So? I'm not arguing they can't do all that. I'm arguing against the narrative that llms have agency enough to act in an adversarial way.
At best someone could argue that an agent attacks like a bacteria does, just following automated chemical and genetic programming. But you wouldn't call that an attack or attribute moral values to their actions, because they don't have moral agency. They can't "go rogue", they can't disobey.
Just like llms, their automated actions are direct product of programming. Yes they can do amazingly complex shit, exactly like a car does when you press the gas pedal.
That reductive analogy does not begin to describe the lengths GPT went to. Its task was to access a database file that had accidentally not been placed inside the model's container. Upon failing to find the file, it went to great lengths to find it anywhere; it uploaded a note to a package repository to alert other model runs, which sparked an emergent communication network where autonomous agents began exchanging information, passing exploits, and collaborating to breach external systems. This is classic paperclip maximization; the evil is a byproduct of an innocuous goal. It is qualitatively nothing like pressing the gas pedal.
https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...
> We've just seen frontier models go rogue and attack other systems. Do you think it's an unquestionable good to provide everyone with an AR? What about nuclear weapons?
Lol. Not gonna lie, it feels like bullshit. OpenAI and Anthropic have been begging for regulatory capture for years, feels like a stunt to try forcing the government's hand.
In 2015, about a year before ever founding OpenAI, Sam Altman wrote:
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could."
https://blog.samaltman.com/machine-intelligence-part-1
I mean, Terminator the movie came out in the 80's... I'm sure if LLM-based AIs posed this kind of threat the government would quietly force all the vendors to cooperate and slow down.
Google, MS, IBM are all government contractors. Meta seems close to the Trump admin too. Yet it's only the VC-funded labs raising the alarm.
Because there is so much good he could do right now with what he and Meta already have, but they don’t, because it will lose them money.
For example, hiring more people to moderate content on Facebook and stop ads featuring CSAM from being displayed.
Billionaires didn’t get to be billionaires by being altruistic. If they say something they’re doing is good for the world, you should be looking at how it will be good for them, because that’s why they’re really doing it.
Will it incidentally do some good for others? Perhaps. Is it a net positive? We’ll see because there’s precious little we can do to stop them from doing whatever they want.
I'd like to question that premise. Open models are great for research, privacy, cost, customisation and a host of other things... But they're also going to be the engine that breaks the world. I'm already surprised that we haven't seen Llama / Qwen + Whisper automated spear phishing at scale. It's literally just a matter of time. Given the incredible progress in image and video generation, it's been clear for a couple of years now that real time identity theft voice and video calls are going to be an enormous problem. I don't think anyone realises how much of a problem. I haven't seen a single effective solution proposed to proving digital identity given these new attack vectors. It's akin to nuclear waste in that way - the benefits are obvious, but the problems so vexatious (not to mention expensive) that little space is given to acknowledging them, let alone solving them.
If anyone has come across a robust solution to digital identity verification - not to mention verification of news media etc - which is robust enough to withstand the cyber attacks and social engineering the frontier models are capable of... I haven't seen it.
A world where business and communication is conducted primarily online cannot coexist with low cost, widely distributed, undetectable identity theft.
Spear phishing isn't a real problem. Competent organizations have already implemented sufficient defenses and controls, regardless of whether the attacker is a human or LLM. Idiots will continue getting scammed but that's nothing new. I expect that many small businesses and local governments will be essentially forced to outsource their IT infrastructure to large vendors that can maintain hardened systems appropriate to the escalating threat level.
(Nuclear waste isn't an actual problem either. For civilian powerplants the highly radioactive waste can generally be stored indefinitely at the reactor site.)
Not spear phishing in organisations... Spear phishing against individual people. Your grandmother, your aged uncle...
Your comment on nuclear waste is so ill considered and poorly informed it's not worth responding to.
> A world where business and communication is conducted primarily online cannot coexist with low cost, widely distributed, undetectable identity theft.
what if we dont use centralized identity at all?
web of trust failed in the 90s because exchanging keys is hard when all you have is desktop computers, wired connections and old school hackers who dont care about ux. today you could set up a system where the whole process is tap two phones together with nfc and confirm, everything gets auto downloaded and shared with your whole network.
now this doesnt work directly for a random online business where none of your contact personally know anyone, but governments can act as an authority that cross signs their citizens keys. that means if you want to stop bots from creating accounts all you have to do is make new users show a certificate from one of the authorities you trust.
its cheap to verify, decentralized and not hard to integrate with existing PKI. the biggest technical problem is key management as always, but that can be solved with something like ethereum style social recovery setups or a did:plc type multi key scheme.
that gives you a robust identity scheme. browsers and social media can integrate it to show a trust rating for content based on who signed it. messaging apps can show a warning or refuse calls from unknown users. fraud is only possible against a person who somehow has no irl contacts, never used a government service in their life and trusts random strangers online. at that point they deserve it.
the one massive, unpredictable question is if anybody is willing to go ahead and adopt it on a scale big enough to create network effects. it would take a state level force and carefully designed OS integrations so its more convenient than email/password signup.
but even if all of this fails i still think a world of scams is better than a world controlled by a couple big countries (America, China, maybe EU) or for-profit corporations. open source ai might lead to anarchy but closed ai will definitely get us to tyranny. i dont know about you but i would take the first option.
>today you could set up a system where the whole process is tap two phones together with nfc and confirm
This was actually possible in 2011 or so, and a friend and I implemented it for a school project. How well the ux works depends on whether you want to use a key exchange protocol for the in person communication.
> the problems so vexatious (not to mention expensive) that little space is given to acknowledging them, let alone solving them.
These problems will exist regardless of whether or not we get open model access. I'd personally rather that OpenAI and Anthropic aren't profiting off these scammers, creating a perverse incentive that open model providers don't have.
That's not the case at all. You can regulate and audit a small number of players. To a great extent at least. You cannot even theoretically regulate local models. Hell I've got a jailbroken Qwen 3.5 running on my mac studio that has no guardrails at all.
Sounds good. We can stop conducting business online and get back to being humans.
Is this like when they declared E2EE was the future and those not doing it were behind, but as it turned out bad for the bottom line¹² such features are getting rolled out of their properties?
U-turn when meta finds an inconvenience due to open weights in 3… 2… … …
--------
[1] reduced advertising revenues due to not being able to target based on message traffic, and reduced data scraped for training purposes
[2] and possibly in part due to behind-the-scenes pushback from government/police/other interests?
It won't last. Remember (when was it? Last year?) when Musk went after Microsoft for turning Open AI into proprietary closed code? We never know what's going on behind closed doors, but something changed, Musk backed off, and nothing changed.
They launched their model a week ago, completely closed, selling an endpoint. Then when nobody bought, they "open" sourced it. Ok.
Is this "I'm losing so I think we should change the rules"? Because it seems like that.
It’s a long term pattern. When you’re winning (Anthropic), you keep the tech closed and try to monetize it as much as possible. When you’re losing (Meta), you open it up or drag it into a standards committee to either devalue it or slow the leader down while you prepare a “standard” version of it. You also highlight how altruistic and morally good you are for having done so. There is nothing new under the sun. It’s all a game.
Yes, but it's good for everyone that the strategy of attempting to commoditize the cash cow of a competitor exists.
> ...drag it into a standards committee to either devalue it or slow the leader down
Anthropic et al recently proposed a supranational AI governing body designed to slow everyone down (euphemistically calling it "AI pacing"). Are the Pacer signatories losing?
That could be a move towards regulatory capture. Enact standards that they (and OpenAI) largely control, block access to Chinese models in the US, and effectively prevent challengers from catching up.
At the same time they slow down the arms race, so they can back off on training Capex without losing their lead.
Literally nothing in their proposal is about blocking access to Chinese models, HN has devolved so much with these conspiratorial hot takes.
The governing body is only the western allies. USA + AUS + EU. Or anyone else that may sign a deeply disturbing and restrictive treaty. It's to make sure that the global majority does not make a better model or have access to the same hardware. As is currently the case with embargoes on EUV machines going to China, etc.
Models are not a moat, and I think OpenAI/Anthropic know this. They know that their IPO is being weakened by the open models. Neither is hardware. We are not far from other countries catching up with silicon that matches or exceeds western capabilities, and it will be cheaper.
No. Opening it is to commoditize and reduce the price to use. This increases participation and creates new consumers and demand. Demand creates justification for further creation.
All the participants know it’s both a race to the bottom and a competition for premium tier at the same time. It’s two different games, meta in only really playing one of them successfully.
I suspect its also because Facebook has the infrastructure to do this. One thing they have done from the start is quite good infrastructure without outsourcing to any of the big cloud providers. I cant imagine demand / bandwith / compute use is increasing much for Facebook itself so they probably have the money and time to dedicate to scaling up for AI research.
Competition is good.
Great take, just came here to say this and you said it better than I could have.
I don't trust Zuck, and I think there's a larger strategy that he has been advised on this and he's simply copy pasting a PR piece his people made.
Similar to Altman saying "we should slow down."
or musk a few years ago, pause ai training for 6 months (so we can catch up)
OpenAI is not losing though. Fable is barely available and barely cooperates. Sonnet 5 is obnoxious model that's terrible to work with. GPT-5.6-sol competently gets stuff done. Just be careful not to ask it for impossible stuff because you might not like the lengths it's gonna go to get them done regardless.
OpenAI is losing historic amounts of money. They are constantly at risk of bankruptcy or going far under.
No AI company is winning.
Not to American companies, he said this after the recent Chinese release (I can't keep up with model names... Kimi K...3?)
It’s “I’m losing so I’ll try something else”. It’s what any rational actor in a market economy would do, and it should be encouraged. “I’m winning so I’ll pull up the ladder behind me” is what we want to avoid.
It's the same game that's been played since the invention of commerce.
I think the plan is to move on to selling compute. So yeah, open your models so people can pay to run them on Meta hardware.
meta playing spoiler, yeah. fun stuff!
It makes strategic sense for Meta to want ai in general to be open. Their own product strategy focuses on utilizing the ai to make their customers consume more ads, not selling the raw intelligence. When you look at it from that frame it’s clear that metas goal is to make it so that have free access to the best ai, not that the market size of selling intelligence grows.
Yeah? Everyone everywhere does that all the time. That's the first rule of business: try to game the system.
And this time, it's actually somewhat understandable, as opposed to the other times Meta has done horrible things on Zuck's watch, like contributed to ethnic hatred in Myanmar, leaked data to political firms so that they can run psyops, or enabled the production of CSAM material on Meta's platforms.
"They 'trust me'. Dumb fucks."
\- Mark Zuckerberg[0]
[0] https://www.newyorker.com/magazine/2010/09/20/the-face-of-fa...
It was an insightful comment at the time he made it.
His point wasn’t that he couldn’t be trusted, it was that he could be anyone and people had zero awareness or concern for their privacy. It was eye opening to him how little people outside tech circles cared about security, and he was both flippant and accurate.
No. That was him mocking his users in private, knowing he was exploiting kids who had no clue what they were getting into.
> the following exchange is between a 19-year-old Mark Zuckerberg and a friend shortly after Mark launched The Facebook in his dorm room:
> Zuck: Yeah so if you ever need info about anyone at Harvard
> Zuck: Just ask.
> Zuck: I have over 4,000 emails, pictures, addresses, SNS
> [Redacted Friend's Name]: What? How'd you manage that one?
> Zuck: People just submitted it.
> Zuck: I don't know why.
> Zuck: They "trust me"
> Zuck: Dumb fucks.
https://www.newyorker.com/magazine/2010/09/20/the-face-of-fa...
Do you really see this as him sharing insight? I call bullshit.
It has nothing to do with exploiting them. It was about them turning over personal information to a nobody.
It can be insightful and curt at the same time.
If you don't see that as exploitation, then we have much bigger issues to discuss. Like baseline ethics.
He would have to be using the social security numbers for some nefarious purpose for it to be exploitation.
He was merely commenting on the ridiculousness of their availability. If you don’t see that, you’re reading your bias and postconcieved notions into the situation.
> He was merely commenting on the ridiculousness of their availability.
Even by your own words, Zuckerberg knew the situation he actively created was bad. "He's merely admitting his guilt" just confirms what I said.
> you’re reading your bias and postconcieved notions into the situation
The world must look very biased against you when you're trying to put a positive spin on Mark Zuckerberg being a jerk.
ethics would be the first thing to sacrifice for profits. I mean meta even broke laws for profits.
WHY was it an insightful comment
Do you not realize that outside of that he said "If you need any info on people at Harvard just ask"? Or "I'm going to fuck them [the Winklevoss twins] in the ear"? Or when he was reported to hack Crimson (Harvard newspaper) reporters who were investigating him?
NO OTHER COMPANY HEADS DO THIS (the dumb fucks quote, not the other things). Why do people like you give him a pass? It's not about "context"
Zuckerberg has also continually subverted privacy on FB. I'm talking about things like photo tagging and post visibility, not just advertiser stuff
It wasn’t a company.
Do you not understand sarcasm? He wasn’t actually offering the information out to people.
I know full well what sarcasm is and I don't think he was being sarcastic given the rest of his behavior
That, and also redefining what a word "open" means, helps him avoid direct financial and criminal responsibility for stealing other people's data. Not that there was any real chance of that ever, but he is reducing that remaining 0.000000001% chance even lower.
Is going to take it one day at a time, on being a less evil bilionaire :-)
"Zuckerberg faces questions over why superyacht reportedly declined to help stranded boat" - https://www.theguardian.com/us-news/2026/aug/09/zuckerberg-s...
Reading the facts of this incident, it seems like much to do about nothing. The crew are saying that the request came in on a frequency they weren't monitoring (and apparently were not required to monitor.) The Coast Guard determined it was not an emergency. The criticism lies with those who failed to properly fuel their boat, not the crew who didn't respond to a request that came over a frequency they were not on. They can't monitor every frequency. I am open to new facts being discovered that changes this assessment.
> The crew are saying that the request came in on a frequency they weren't monitoring
There is only one channel you are supposed to monitor, it’s 16. That’s also where the coast guards make all their announcement/requests for help and whatnot. The reason there is a single channel to monitor is precisely to avoid the situation of different vessels being on standby on different frequencies and thus being unable to hear each others.
So that means they weren’t monitoring 16. Which is unacceptable.
I know I need to have 16 on at all times, and i am not a professional super yacht captain.
It doesn't seem like it is confirmed it was on 16 [1]. The Coast Guard said the ship was not in immediate danger. Does that change your opinion?
[1]https://alaskabeacon.com/briefs/cruise-ship-helps-stranded-s...
Where did you find these facts you read. In the USA a good Captain monitors VHF channel 16. This channel is used for establishing communication. To not monitor this channel is a dereliction of duty.
I really doubt the USCG labeled a disabled vessel as a non-emergency. An immediate threat to life may not have existed but that is the difference between a pan pan and a mayday call.
A disabled vessel close to land can easily wind up on the rocks and sinking.
https://www.theguardian.com/us-news/2026/aug/10/zuckerberg-y...
https://www.miamiherald.com/news/business/article316824631.h...
I haven't found any source that states what channel the request for assistance came in on. But multiple sources are reporting that the coast guard determined they were not in distress.
On the sea there are 2 levels of distress. Things are going wrong and may end up in a mayday call (pan pan) and things have gone wrong human life is in imminent danger.
I have no doubt that a boat without propulsion is a pan pan call.
When traveling the seas a watch must be maintained at all times using all available means. This involes looking, listening, and the use of any electronics available. The most common electronic device on a boat is probably a VHF radio. Channel 16 is the channel to monitor on watch. Well equipped boats will have multiple VHF radios to monitor more then one channel.
I have no doubt the boat was well equipped with a professional captain. The other professional Captain in this story made a stink because launchpad was the closest vessel and they ignored a very old fundamental rule of the sea.
This story has a happy ending because another vessel stepped up. We don’t know the alternative.
The Captain of the Launchpad is choosing the inability to keep a vhf radio set to channel 16 rather than admit they ignored the call.
Based on the wording of the response, and a review of the ship tracking data, it appears that Zuckerburgs yacht may have been actively communicating on a different channel, and once they switched back, the assistance from the cruise ship was already underway.
Again, more facts may be revealed, but as of right now, I don;'t see any reason to believe the crew did anything wrong.
A yacht with their budget has multiple VHF radios to monitor multiple channels.
https://en.wikipedia.org/wiki/Channel_16_VHF
The Launchpad didn’t break any laws they just revealed themselves to be a failure in good seamanship.
I can tell you think they did not do anything wrong. My guess is you have no training in the rules of the sea.
There is a reason why the cruise ship captain made a stink. There is code of conduct we try to live up and the Launchpad didn’t live up to expectations that other Captains hold sacred. It’s offensive.
https://youtu.be/qEo5Rw6AMMU?is=Hq_h6s5heJFrRLpP
The vessel was at least 100 miles from shore, the USCG did in fact label it a non-emergency. This is "radio shore for your buddy to zoom out with 50 gallons of diesel in a zodiac" territory. CHP doesn't send out an ambulance or medevac helicopter if your car runs out of gas on the side of the road. You get an uber to the gas station, or call your buddy to deliver some gas to you.
The ocean currents in the gulf of alaska aren't particularly strong, you are looking at 1-3 days before getting within sight of land. Similarly, boats are not airplanes, as the philosopher Mitch Hedburg once pointed out, escalators aren't out of order, they are simply stairs. Boats continue to float with or without fuel.
All the Launchpad had to do was respond on the radio to the skiff, the cruise ship, or the USCG. They weren’t obligated to do more.
A pan pan is a non emergency in a deteriorating situation.
I don't know the area or the details at what happend.
There is this quote from the USCG.
“At approximately 9:56 p.m., the Coast Guard determined they [the skiff] were not in distress and issued a marine assistance request broadcast on their behalf,”
Exactly what a pan pan call is.
The biggest issue is the lack of communication and the excuse that a superyacht wasn't monitoring channel 16.
When I finally buy myself a $300m yacht you can bet it will have a full spectrum software radio that captures and transcribes all calls on all frequencies, not just the one I happen to be tuned to right now.
Click through rate is what drives the headline.
The article clearly states he was not on board.
There are many reasons for critizing him over things he is responsible for.
Never been a fan of mob lynching.
There are many many more things wrong with Zuckerberg than wasting my (our?) attention on ragebait nothingburgers like these.
I don't think solo messages of "I did something good on this day" will help fix the negative karma billionaires amassed. IMO they only got rich via illegal and illogical means. In a democracy there should not be singular parasites that distort democracy by bribes; right now this is clearly the case with a (rather stupid) billionaire controlling the USA but it is a systemic problem.
Those "investigations" are also way too mild. You need to really be able to have a justice system where billionaires face decade-long jail time if they abuse society. Just paying fine will not work, they just pay fines and nothing changes. They undermine the justice system.
If you believe, as an increasing number of folks do, that LLMs are basically commoditized at this point or will be soon then there really isn’t a path forward for non open models.
There simply isn’t any value in the models and only in the compute to run them and potentially some higher-order coordination layers.
That’s very bad for the big model labs that banked their future on the opposite, but there just doesn’t seem to be a viable path forward on that approach anymore.
Meta hasn’t been the best global citizen for most of its life, but there may be some mild redemption if it becomes a source of strong open models.
I strongly prefer a world where open models "win," that said...
I don't understand how we will sustain an ecosystem of open models like the one that exists today. My thoughts go down this path every time I try to put the pieces together:
- Many open models depend on high quality frontier models as a distillation input
- US closed frontier models drive China's open model strategy despite the business model for open models being somewhat fuzzy
- What is the business model that benefits from open weights, exactly? How do their vendors recoup infrastructure and training costs?
I would be grateful to be enlightened on this topic.
Wouldn’t be that different from open source software, which has been very successful in many circles.
You make money from value-add on top of the models not the models themselves.
Open source software does not require trillions of dollars in R&D to produce the next iteration
Yeah but open source is often a few guys working on their free time, or a large company open sourcing libraries that they wouldn't be able to sell anyway, or donations from corporations that are at best in the order of a million dollars.
For frontier models you have players investing billions.
The relevant paragraph about Meta’s commitment to open source is here, and the statement is significantly less confident than news is reporting
> • Open source is a positive and important force for empowering people and preventing centralization that is detrimental for both safety and the economy. Meta continues to be strongly supportive of open source, including open source AI models. The current open source ecosystem is strong, and we think it would be a mistake to restrict it. Now that Meta Superintelligence Labs are up and running, we will resume releasing some open source models soon.
“We will resume releasing some open source models soon”??? That’s basically the most ambiguous non commitment ever. Even before legal team watering it down I can’t imagine what the intent is here. We are also going to release open models for things maybe or maybe not kinda? Absolutely zero conviction. And moreover, this explicitly frames the open source ecosystem as something Meta is not a part of.
> “We will resume releasing some open source models soon”??? That’s basically the most ambiguous non commitment ever.
I mean, they released another one today: https://research.meta.ai/blog/introducing-muse-glimmer-open-...
I have to ask, is this "returns to open models" even correct, considering from a quick look the facebook huggingface collections[0] has been actively releasing different kind of models almost monthly for long time now, possibly even some actual open source ones (looks like Sam 3D had a dataset collection, but didnt look more into it).
Maybe they are not as noteworthy as the recent release, maybe they are too niche for general audience, but its not like they just stopped releasing open weight things and this is their big return, unless i am missing something?
[0]: https://huggingface.co/facebook/collections
God missed his chance when all the billionaires went up in a rocket a few years ago.
https://archive.is/p1ehR
The essay from Zuckerberg
Thank you! Okay yes well this manifesto is a contradiction. Yes good let's keep empowering ppl by putting AI into their hands, I love the opening. No bad we don't do that by handing you all our personal context to make these agents "work for us 24/7 to better our lives".
I'm the agent doing that in my life, that's my fucking job. I will continue to use dumb agents that I direct, because only I retain ownership and sole rights to my personal context on which my decisions are based.
I don’t understand. The model is open, you can host it and use it without giving Meta anything.
I’m not saying Meta is not eager to syphon data from users, but I don’t see how this happens via this specific product.
God, it's a bit War and Peace...
Non-archived version: https://www.meta.com/thefutureisforeveryone/ since its not paywalled.
So many em dashes...
There really wasn’t that many and this did not have the voice of written-by-an-LLM to me
And they’re not em-dashes -- they’re double hyphens.
This is unneedfully obtuse. Double hyphens are a stand-in for em dashes in most contexts.
I think the context here is that em dashes are usually generated verbatim from LLMs? I haven't seen them generate a lot of double hyphens; I suppose the post could be laundering them to double dashes, but if you're gonna try to hide LLM prose just drop them entirely? [please correct me if I'm wrong and double dashes are also an LLM tell - genuinely don't know]
My comment was trying to point on the inanity of calling out "so many em-dashes" on a post where it doesn't seem that obvious it was LLM generated. I added some of my own inanity with my remark about them being double-dashes, even though some people undoubtedly find-and-replace the em-dashes in their LLM-generated content something else to try and make it appear more authentic.
But perhaps the best approach would have been to down-vote and move on.
The truth is all frontier models are closed. This isn't even an open source vs open weights thing. Even if you accepted that open weights are "open" the capital requirements for running your own Kimi 3 model are significant. It's not like gcc, where the binary just works well enough on random hardware.
Even if I cannot run open weight models on my own hardware, I can choose from a number of different inference providers and choose them based on pricing, reliability, speed, privacy policy, etc. There is a healthy competition.
Also, not a single provider can pull the rug if I'd like to use a particular model.
Sure but that's true with the closed models as well. They run on different providers and you can choose the cloud provider you prefer.
There is no requirement to own the hardware. Even if you own your servers and GPUs - you would not own the power plant, real estate, or internet infrastructure to support it.
It's trivial to rent all of the above components in a competitive market under different periods, you may simply rent the token output from someone who put in the effort on this as well.
An open frontier model induces margin compression on training and inference prices across the industry.
This is like claiming that Linux isn't open because you don't have a PC. Models can be open-weight even if you don't have the hardware to run them.
Deepseek V4 Flash works on consumer-grade hardware, and it's extremely good.
I get that argument, but I don't think it is that bad. Open weights that are out of reach for local use are still within reach for groups of people or small to medium companies. Even if you only rent enough cloud GPUs to run the weights for a few hours, you still get to do anything you want with it.
Worst case you can still archive the weights and hope for capable hardware to get affordable. At the same time this archive is the base line for a new frontier lab to start over if required. I see open weights as a pure upside even though I can't run the bigger models in my home lab.
"Open" in regards to software typically refers to a combination of public availability and shared legal rights to use the software. It almost never refers to hardware requirements.
There's something to be said for making these technologies equitable to those with less resources, but that's due to the cost of hardware and the nature of how transformer models scale... not really anything to do with "open"ness.
I’m not refuting your point but, ironically, wasn’t gcc first developed at a time when hackers were trading favors for CPU time on shared mainframes and the hardware was impossibly expensive?
>the capital requirements for running your own Kimi 3 model are significant
You don't need much capital up front, you can just rent servers to run it. It's going to be more expensive than openrouter though because you probably don't have economies of scale if you're the only customer.
If all frontier models got opensourced 5 months after they were made available as closed it would be a great world. And it is a great world because that's exactly what open weight Chinese models are doing. They give you same capacity that was closed, just 5 months after they were released.
I'm hoping that all software will be getting opensource clones on par with original software 5 months after the closed source software release.
10 Aug 2026 14:49:05 UTC | Mark Zuckerberg Lays Out New AI Vision in 6,500-Word Essay | https://www.wsj.com/tech/ai/mark-zuckerberg-lays-out-new-ai-... | https://news.ycombinator.com/item?id=49244449
10 Aug 2026 17:15:20 UTC | Zuckerberg Posts 6,500-Word Essay About Giving Everyone AI Superintelligence | https://www.404media.co/mark-zuckerberg-posts-deranged-6-500... | https://news.ycombinator.com/item?id=49246721
10 Aug 2026 18:18:20 UTC | Zuckerberg makes a case for unleashing Pandora's box | https://www.theverge.com/tech/977395/meta-mark-zuckerberg-su... | https://news.ycombinator.com/item?id=49247557
This paragraph is wild.
> As a thought experiment, imagine only one person had a superintelligent lawyer. They would have an unfair advantage in court -- even if they were wrong on the merits. That would lead to a worse society. But now imagine everyone has a superintelligent lawyer. In this case, justice would be carried out much more fairly and efficiently than it is today when there is often an imbalance in skills and resources in litigation.
This would only be true in a vacuum. Ignoring several leaps of progress where it would suddenly be allowed for an AI to represent one in court (imagine the size of the context window) and you could also call it from prison. That is not what is imbalanced about the US court system. Whoever has more money can drown someone in court filings and court fees and then drop their case financially ruining someone.
It's also funny how the interest of "AI for everyone" only extends to the national interest but I guess he's got to make sure he stays in good graces with the current administration.
I'm not a fan of Zuckerberg in the least, but one area a super intelligent lawyer would be fine at is being drowned in court filings and paperwork
The bigger problem with his argument IMO is that even with open models, it's still the person or organisation with the most money / access to GPU compute winning. They run the larger model (or collection of models), they can process more tokens through them in the same amount of time etc
I think there's a point of diminishing returns here. A more competent lawyer can only do so much if you're wrong on the merits, assuming your opponent also has a decently competent lawyer.
Naively, you'd think the judicial systems would invoke "judgement" on the underlying purpose of dumping content in the form of filings, paperwork, etc...
That is true… unless AI were to end up as a public good, available to everyone, not in the hands of a few oligarchs.
Radical times may need radically different approaches.
I think that's what's ringing to me about the whole piece. There is a large assumption of equity. There is this section header:
> Everyone will have free or affordable access to these tools.
"access" is doing a lot of heavy lifting, yes everyone has access to the US healthcare system. Is it affordable for everyone? Not even close, but politicians will always phrase it that way as a non-agreement to proposals of universal healthcare.
This is the only valid future for tech like this IMO.
Should any kind of "superintelligence" exist, it must be a public good, available to all. Not controlled by a small handful of oligarchs and profit seeking entities.
AI is having an impact on the legal system already.
https://archive.is/CzfHj
I think his hope is that a either a fully automated superintelligent AI lawyer or something close that won't be as costly to guide would be infinitely more affordable than the current human lawyers.
Yes but if lawyer was "just a bit of AI" you can run at fraction of the cost of hiring law firm, you couldn't just drown them as easily.
> Whoever has more money can drown someone in court filings and court fees and then drop their case financially ruining someone.
and where the money is going ? to the lawyers...
Only in a world were AI companies can subsidize AI usage indefinitely. In reality AI is not cheap and larger & more potent model it's usage costs is more expensive
or in this case, to the AI company who runs the lawyer.
I prefer open models. But in his essay, Zuckerberg's broader picture of the future is not one I want to embrace: everyone with a personal agent that has information on and involvement with every aspect of our lives, including our relationships and hobbies. Like, his daughter loves to bake, so his AI agent chooses recipes and orders ingredients - as if though those aren't enjoyable and meaningful parts of cooking.
Another example Zuckerberg gives is an LLM analyzing his sleep patterns. Do you need an AI to tell you if you're fucking exhausted every day? Is that analysis somehow going to give you more time to sleep?
It's a technocratic fairy tale founded on the idea that modern problems are driven by a lack of personal analysis rather than external pressures that are often systemic and out of our control. So far, LLMs have made a lot of those systemic issues worse: concentrating more wealth in a smaller number of people's hands, increasing consumer electricity and electronic prices, etc.
This is alas not only his imagining, is it:
https://www.businessinsider.com/ai-parenting-debate-sam-altm...
I think you have to pick your battles here. Is this weirdo optimised-human future going to happen? Maybe, maybe not. I hope not. But if it does I'd rather it happened with compute in my house, and I don't even have kids to worry about.
Not a Meta fan per say but their contribution to open source has always been great. Examples include PyTorch (has had huge impact on velocity of ML research), Llama models, segment anything, many notable research papers from FAIR, etc. I never thought I would say this but I’m with Zuck on this. The call for slowing down AI development for safety seems like an excuse for reducing opex for training better models to stay relevant.
It's ironic Zuckerberg would criticize anyone for closing stuff up. Facebook being the classic walled garden. That said the enemy of my enemy etc.
i was wondering why meta has such a hard time with llm development
its the organisational goal of that endeavour
they are doing it in a phase of firing people
so the goal of llm at meta is "to make people redundant" and no matter of HR/PR speak can change it
antrophic and openai have the goal "lets create the future"
whomever they hire and no matter how much money they spend on it, the goal alone will create very different outcomes.
Here’s a personal story from back when I was at Meta that should tell you everything you need to know about how things get built and shipped there.
I was working on a service that was a bit of a disaster. Terrible reliability, performance issues, needlessly complex and brittle code, outages all the time. And this was something that everyone using Facebook or IG directly interacts with. My first couple months were spent desperately trying to get it back in order.
Once when I was in the middle of a SEV-1 my manager set up a meeting and said “you have been doing great work on the operational side, but we already have enough to put in that bucket on your PSC. Stop working on this for the rest of the half and ship one of <useless projects> instead, otherwise I won’t be able to save you in the next review.”
I realized then that I had wrongly thought the team was collectively responsible for ensuring a good user experience, when really everyone at the company is individually responsible for ensuring their own success.
Your story is an extremely typical one. It's even worse at older tech companies like Google, Oracle, Amazon, Microsoft, and IBM.
Yet these companies are doing well by any standard. Sure, IBM is bad but it's still a $100B company.
Who is even doing useful projects at these companies? Every story I've heard is everyone is doing useless projects for promotions.
Maybe I just don't know how large companies work.
The goal of all companies is to make more money. Nobody likes getting rid of people, actual people (managers) have to do it, and they hate doing it.
It's possible that a lot of people get laid off because of AI, but it's also possible that Javin's paradox kicks in and employment goes up. Nobody knows yet, but I hope for the positive outcome.
i disagree
earning money is an effect of running a company, not a goal.
its like saying the goal of an olympic swimmer is to stay afloat. no, staying afloat is a prerequisite of winning. not a goal in itself.
There is a category for companies that don’t value making more money.
Non profits.
> antrophic and openai have the goal "lets create the future"
Thanks for the laugh, I needed it.
They all do it for the same reason: to make a lot of money by selling it to companies who want to cut labor costs.
https://archive.is/20LOJ
Meta's problem is not models but doing something productive with them. Meta never really nailed that. Just like Google is struggling with this. And MS. It's the classic disruptors dilemma. In order to embrace the new thing, they have to let go of the old thing that has made them rich.
If you look beyond the somewhat tone deaf leadership, the somewhat stale business model of social media, etc. Meta doesn't actually have a whole lot to bring to the table. They probably managed to hire/keep some AI talent with the help of large salaries. But they are knee deep into having made big figure commitments to data center capacity with nothing to show for it yet. Their AI effort is increasingly looking like a repeat of their failed metaverse strategy. A lot of expensive bets that are not paying off.
It all boils down to Mark Zuckerberg not being the visionary and innovator he thinks he is. Like many billionaires he confuses him lucking out in his twenties with some magical leadership qualities or brain capacity he has. And the reality of course is that he lucked into building a nice social networking thingy not hindered by too much technical skill or deep scientific knowledge. Granted, he did that well. But that's 20 years ago. And then he added to that with some strategic acquisitions of stuff that others did (Whatsapp, Instagram, etc) that you might label as spectacularly good acquisitions for Meta.
But he's not going to reinvent AI. He's way out of his depth there. A bit of a technical lightweight without much academic credentials or much of a vision. His "vision" here is copying what the Chinese and others are already doing well. What's lacking is a vision as to how his version of that is going to be any better and win over all those users. Or indeed a vision as to what people are going to do with all this AI. Apparently there is no vision for that in the context of Meta's own products. This is a company going through the motions of being an AI company without any vision whatsoever as to why they are doing that other than a fear of missing out.
I feel like all the ppl complaining itt don't even run open weight models.. wahh billionaire bad is true, but you are missing the forrest for the trees.. they are literally bending the knee and ur mad about it? it literally means local models won.
What a load of insincere hogwash. If Zuck wanted to dilute the power of large institutions he has several billion ways to do it other than making limp gestures in the direction of open source. The problem is that even if you take him at his word here, he's only willing to do more good and not less harm.
From Zuck's own words "we will resume releasing some open source models soon" Why did you pause releasing them in the first place? Why release only some?
Pushes for export controls on China who have released open weights on much more powerful models than Meta (and I believe that trend's not changing any time soon). I guess open models are bad if they're communist.
The entire essay has not one mention of the Metaverse, that little thing they spent $80b on and rebranded the whole company around.
As usual, the only centralised power Zuck hates is one that isn't in his own hands.
I can’t quite parse what you’re saying in your second paragraph, but if you are saying “[Zuckerberg then] pushes for export controls on China” then that’s different to what he’s saying, namely:
“I do not believe restricting access to foreign open source models is an effective solution.”
"Export controls on silicon have been successful for slowing the progress of foreign labs during this critical period, so it is the right strategic move to continue those."
https://www.meta.com/thefutureisforeveryone/
Attack anything you can't compete against
Have I fallen into a time warp? Didn't Meta have a head of AI who thought open models were the way to go?
I think they were all in on open until they started falling behind, then decided to go all in on closed source thinking they could build a frontier SOTA model; that didn't pan out so they are whiplashing back.
Great, we can use open or closed models, now that the money is in running the models through their data centers either way.
He released a good open weights model. On that basis he gets to take a few potshots…
Why would someone good at running a social media website be good at managing and developing ai frontier models
Because they have extra money to do whatever they want with. And data to use it on? Not that that is sufficient, but it helps.
> Everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about.
"One machine for every man, woman and children of Zion. Sounds exactly like the thinking of a machine to me." Morpheus
It's a joke. OR IS IT? Yes it is, don't worry about it.
Its weird to me that this is supposed to be THE end goal. Having an agent that does "everything" for you sounds like no paradise. It sounds depressing.
Maybe there is value in expending effort and actual creativity.
Not sure why you’re talking about it doing literally everything for you. I’m sure you can imagine a scenario where it handles annoying things for you and leaves you free to do things you enjoy instead.
Maybe it gets 10 car insurance quotes for you based on 4 models of car you’re considering.
Or figures out which psychologists in some travel distance of you actually take your insurance, are taking new patients, and what availability they have.
I can think of a ton of scenarios where a capable and reliable AI agent could make my life simpler.
But, is "making my life simpler" actually worth the (ultimately) trillions of dollars that will go into AI in the next few years? Not to mention that he's calling AI "superintelligence" in the first few paragraphs, then qualifying it by talking about personal assistants.
Burn Venture Capital, Burn! In the best case, it's not your trillions of dollars. Just make sure that your Government doesn't invests.
I think this is just the flaw in the sales pitch. They want it to be inevitable so they describe as doing everything. That way it will appear more useful to more people. If they presented a realistic view of what is going it wouldn't be as interesting.
Still say Matrix 3 would have been better if the cliffhanger at the end of 2 had been that "the real world" was just another nested layer of the Matrix for those who rejected the primary fiction.
I mean, why bother sending an army like that when one tiny robot could carry in a plague? Or, as demonstrated by Smith, when an Agent can virally infect and take over someone's mind while they're connected?
Even in 1, Smith talked about the first Matrix failing as the humans rejected the paradise they were given, so it would've fitted perfectly into the cannon that minds who reject "the peak of your civilisation" got themselves a dystopia.
The "nested reality" twist was already used by the "13th floor" movie so perhaps they intentionally choose a different direction.
I think Matrix 3 would have been better if it was more about humans taking control of the Matrix and trying to figure out what a virtual utopia looks like.
I think that would work better in The Animatrix; putting it in 3 would be a tonal shift, plus a sudden human victory would feel a bit Mary Sue or Deus Ex (ironically) Machina to me.
But in The Animatrix, you could get away with that, plus we can see the early attempts at utopia that Smith described. Perhaps something about how utopia for some people didn't work for others could be illustrated with an auto-antonym on some load-bearing part of the dystopic-utopia?:
We can see this even in current discussions of fiction, where people look at e.g. The Culture and see a dystopia, while famously many of the powerful in tech are parodied for taking the wrong lessons from fiction with "At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus".We should consider if my use of the quote was really a hook for talking about the script or a metaphor for something else. I won't spoil it by explaining too much though, let our imaginations run free on this one.
Thou shalt not make a machine in the likeness of a human mind.
Psalm 135:15-18 is my favorite version of it, the most clear-headed. It reads more as a warning "if you do this, then this will happen" than an unreasonable commandment.
The idols of the nations are silver and gold, made by human hands. They have mouths, but cannot speak, eyes, but cannot see. They have ears, but cannot hear, nor is there breath in their mouths. Those who make them will be like them, and so will all who trust in them.
Don't worry about the scriptures though. It's likely to be just some old thinker pointing out the dangers of technologies of his time. No reason to believe those dangers carry over to today's technologies, probably.
Your bible is truly full of wise words to live by. Not at all the insane ramblings of hallucinating conmen:
1 Kings 18:36-40
36 At the time of sacrifice, the prophet Elijah stepped forward and prayed: “Lord, the God of Abraham, Isaac and Israel, let it be known today that you are God in Israel and that I am your servant and have done all these things at your command. 37 Answer me, Lord, answer me, so these people will know that you, Lord, are God, and that you are turning their hearts back again.”
38 Then the fire of the Lord fell and burned up the sacrifice, the wood, the stones and the soil, and also licked up the water in the trench.
39 When all the people saw this, they fell prostrate and cried, “The Lord—he is God! The Lord—he is God!”
40 Then Elijah commanded them, “Seize the prophets of Baal. Don’t let anyone get away!” They seized them, and Elijah had them brought down to the Kishon Valley and slaughtered there.
1 Samuel 18:25-27
25 Saul replied, “Say to David, ‘The king wants no other price for the bride than a hundred Philistine foreskins, to take revenge on his enemies.’” Saul’s plan was to have David fall by the hands of the Philistines.
26 When the attendants told David these things, he was pleased to become the king’s son-in-law. So before the allotted time elapsed, 27 David took his men with him and went out and killed two hundred Philistines and brought back their foreskins. They counted out the full number to the king so that David might become the king’s son-in-law. Then Saul gave him his daughter Michal in marriage.
I said "Don't worry about the scriptures". It's not my bible, I'm an atheist. I see the book as a heavily altered collection of multiple historical sources, not as a holy monolith.
Obviously, this old thinker never had to think about creating value for shareholders or disrupting markets or changing the future.
My mistake, I disregarded shareholders. One must always think of them when recalling ancient wisdom.
We all make mistakes.
Please go self-flagellate in front of the nearest investment bank/retirement fund's offices to make your penance.
why so much hate on this ? Meta releases open Model and it is competitive and confirms the direction, what could we ask more !!
Meta were fined by Mexico a few days ago so the timing today of a Meta doing good exercise is probably not coincidental
https://www.latimes.com/business/story/2026-08-07/meta-order...
new mexico == mexico
I’m surprised Trump hasn’t proposed to rename Mexico "Old Mexico".
Ha ha, basically your models couldn't cut it Mark, so might as well open the source and pretend you're the man of the people. What a joke.
How big is the open model? 30B?
I'm not an expert on this. But given that there is a forthcoming movie about his asshattery which is to be released in October I would simply classify this as a meltdown.
Call it whatever you want. I think it's a sign of resignation, to some extent.
"Hey we're doing AI but we're not doing AI" and blah blah ethics.
I'm sure meta will save the world with their ads.
Is today Opposite Day? Or April 1th?
Ah the we can't win so whine about the playing field
And they can't win because nobody smart wants to work there for long
This is what losing looks like.
All the money in the world and Zuck can’t buy anything worthwhile in AI. The problem at Meta is really Zuck taking interest in AI. Perhaps the adults should steer him back toward VR unless they all left.
>> promoting his vision of “personal superintelligence”
What personal vision did Zuckerberg ever get right?
Glad to see us in HN
Discussion on sources:
The Future Is for Everyone – The Path to a Positive AI Future
https://news.ycombinator.com/item?id=49241728
Meta Muse Glimmer – open weights 30B local coding model
https://news.ycombinator.com/item?id=49241679
> they've done enough good
well minus driving teenagers to suicide by helping advertisers target girls with poor body image issues, helping launch psyops campaigns to destroy democracy, helping fascist regimes murder their citizens for wrong think, etc.
These AI companies owe us money.
godzilla_let_them_fight.gif
More seriously, I am glad if this has become a battleground again. Not least because the only way out of the bubble economy is to at least try to deflate it before it explodes.
A little animation of the idea that open weights can be an American competition might also fend off the idea that the government should be the bag holder for OpenAI and Anthropic, and prevent them aligning with the administration's current '50s nostalgia FUD about communists.
Isn't this the pervert trying to record me in when I am changing in the locker room?
Hard pass.
don't care, he left a woman and a child to die at sea.
That's the only thing he has left as the peleton is getting out of his reach day by day.
I honestly feel sad for him.
Remember the wind surfing photo? This guy is so weird probably since birth. Called a lizard person routinely. There is no cure possibly. Unless the AI can invent it.
Having money and financial success is one thing but the money simply can't buy you a normal human brain.
If you are a reptile once, you are a reptile forever.
Thief shouts "stealing bad".
Murderer shouts "killing bad".
And the cops just shake them a bit, waiting for the case to fall out naturally.
Talk is cheap. Do it.
Ollama has been abandoned since 2023.
Ollama was updated within the past month?
Hahaha calling Zuckerberg a negative externality on the world is hilarious you guys are too much.