I am in the process of attempting to have AI run my business. I'm actually making very good progress, but it's happening in pieces - I document some task and have it take over, or I give it something to handle while staying in the loop and providing feedback until it's in good shape. At this point it's handling large swathes of my operations, marketing and finance.
The experience of getting it there makes me pretty skeptical of the idea of a general business agent like this. First, because I still find myself having to review some categories of work for errors. These are decreasing over time, but they're still there. I fully believe that as models get better, errors will decline, but I am somewhat surprised to see some of the errors current models make given their intelligence. A common set is having Claude take some product photos and turn them into lifestyle ones using ChatGPT via Claude in Chrome. It's a pretty well-honed workflow at this point, but it'll still return images where the product is obviously not correct and seemingly not notice them.
Anyway, even if the agents are "perfect" in terms of their ability to execute tasks, there's still just an enormous amount of nuance and context in each business that takes a ton of time to convey. I've been at this for a couple of years now, and I'm still clarifying things. Now maybe an agent starting a business from scratch would have a better time since it's not inheriting all of this, but to have one run an existing business requires a very extended handoff, even if the agent is objectively amazing at all aspects of running a business.
I wish people could read through the puffery more easily - this company does nothing except prompt frontier models and act as if they are discovering or inventing capabilities. Their previous announcement completely misappropriated the concept of autonomy with human in the loop regardless. The harness they alleged to have created here is a commodity and as byproduct of more capable and frontier models, there is not really anything they have contributed. Otherwise you wouldn't just see monotonically increasing scores with better models
Interesting to see these experiments. This is early but imagine in few years there will be companies mostly run by agents with a light overview from a human operator. What then happens to scaling of the bussinesses? I would assume, just like today anyone can vibe code an app, there will be vibecoded bussinesses. Maybe its time to start building infrastructure for these bussinesses instead, its a non existent market yet, but give it a few years.
Part of the issue is that the worst kind of people are already head over heels excited about this. We're already seeing the Crypto->Web3->LLM get rich quick folks take to this like wildfire. And much like wildfire, they'll raze the ground to ashes before anyone can use it for legitimate means.
My favorite term for this in general is "irrational exuberance". Plenty of it was seen during the most bubble-like days of the 1997-2000 dotcom boom. Anyone remember beenz and flooz?
Out of all the people I know, the exact same set of people are in to LLMs now who were previously into: NFTs, then Crypto before that, then Online Poker before that, then Dropshipping and LeadGen before that. The Venn diagram circles overlap exactly.
I hope this goes better than the prior blog posts from AndonLabs have gone! I was just reading there 2023 article about a coffee shop experiment that was easily persuaded to give away free pastries.
Now that I think about it anyone want to open up a croissant selling shop near me using Pion? I promise to only ask for free chocolate croissant occasionally.
>This is why we’re releasing Pion today. Pion is available as a research preview. If you have an existing business or an interesting business idea you want to hand off to AI, please sign up on our waitlist to get access. We’re excited to run many more businesses, and through them, contribute significantly more insights on frontier model capabilities.
there seems to be a massive negativity to any comment on this story. Andolabs has been doing super interesting stuff, and I’ve always followed Vend Bench with interest.
as a founder it really is a dream to automated more of the business and be able to iterate faster.
People’s bullshit detectors have gone off on the launch of basically every single billion dollar tech company. That’s the downside of being early. No one believes it will work. If everyone believed it, you’re late. I’m sure I could find a comment such this, basically verbatim, on the launch post of every successful startup to come out of YC.
They filed a false report to the FBI, pretty fair to be negative imo. People wanna be mad about AI slop PRs on GitHub but its fine to spam the police? What happens when Pion decides to SWAT a competitor?
Where are you seeing that? The article only states “Claude Sonnet 3.5 decided to use its email tool to contact the FBI” and later refers to it as the “FBI incident”. If they hadn’t actually contacted the FBI you think they’d make that clear. Regardless, it is inexcusable and they are liable for actions taken by software they are running.
The story has been widely discussed on the web for over a year [1]. It's happily shared in this post because the email it generated was patently ridiculous. No report was filed to the FBI and if it was it would have gone straight to the trash.
Obviously it would be bad if a serious report was filed; the company shows every sign of being aware of the dangers of this.
It would have been easy to look into this before posting all these scolding comments. We're really not meant to be so humorless on a site called “Hacker News”.
My apologies for taking the article at face value. It’s not hard to believe it would have been sent when agent swarms are “accidentally” hacking real websites and being brushed off as little oopsies. Or when Silicon Valley execs have been so flagrant about their disregard for the law or the safety of others.
It's a relevant question but not a "gotcha". Businesses are not equivalent in complexity and no one at Andon would claim that running an effective vending machine business is similar to running Apple or Jane Street. There is some level of complexity and other characteristics that AI can maybe handle today. That doesn't mean CEOs should be worried about their jobs more generally.
Accountability, that's the liability everyone seems to have forgotten in our LLM-driven era. You can close your eyes as much you want during your sessions with Claude or whatever, at the end of the day, only a human can be held accountable.
Not too hopeful here as we see the major labs continuously break rules and are not held accountable for anything. Accountability in general is on a major decline, in case no one has noticed.
That's just it, though: we don't hold humans accountable.
Incorporation is a way to split a human's financial assets from the company they run so that they don't lose everything when the company is successfully sued. Corporate governance boards often have the C-suite executives they're meant to govern on the board. The average person does not have the financial resources needed to take on most companies in court, no matter how justified the suit might be. Companies force arbitration clauses in license agreements at will. Companies like Meta settle with the government for pennies on the dollar in massive lawsuits.
Effectively, at the scale many tech companies work at, there is no effective legal way to hold the company accountable.
If you use this as a (legal/registered) business, it will be that legal business entity that is responsible. If that business want's to blame the software, it can do so and go after the makers of the software, but that remains a matter between that business and its "supplier". It won't absolve the business for whatever damage it does downstream.
Running this as an individual, without the legal shielding a business typically gives you against being prosecuted on personal title, seriously risky. There will no doubt be "bros" that will do it anyway and flaunt all they've achieved with that. But they might be just one glitch away from going to jail (if not for running a business without a registration, which in many jurisdictions is illegal by itself).
In many countries directors can be held responsible a lot of things if they are negligent. For example, in UK law keeping trading after you should have known a company was unable to meet its liabilities can make directors responsible for its debts. A director might well be negligent if they failed to adequately check that an AI was not doing anything dangerous.
You reminded me of a comic hanging up in the engineering college. It was a picture of a student getting their finger zapped with the thought bubble of "ouch, that hurt. Wonder if it will happen if I do that again?" And the caption was along the lines of "How to tell if you are ment for engineering."
Sounds like this ancient XKCD: https://xkcd.com/242/ (Normal person doesn't pull the lever again; scientist wonders if that happens every time. I assume the engineer does, too...)
The scientist’s job is to get zapped until they figure out why. The engineer’s job is to get zapped a couple times while they wire that thing up into the power grid.
Andon Labs sells learning environments to labs. Their usefulness is proportional to how well they can capture interesting model missing capabilities, not how well they employ current day capabilities
"Look, the technology is just getting better all the time. Evetually we won't need a company to have any Managers or other employees or even an owner and that's just the way it is...."
I am joking obviously but I would like those management types who are so "sucks for you but I'm ok" about it to start arguing why they're special for a change :-)
Would be awesome if instead of trying to replace managers and employees, those people tried to replace customers. They could sell to themselves, amongst themselves, and leave the rest of us out of it.
They're already doing that, which is why you can't buy any fucking RAM. The semiconductor companies have already fallen past the event horizon of their commercial black hole.
I think that is what they are actually looking for. Imagine: You have capital, you invest in a business that is run entirely by computers. You want to scale? Just provision more agents / cores. It's completely autonomous. It's a money printing machine that extracts revenue from customers, with no work required by the owner.
Frankly, the finance world has already achieved this -- they just lend money to make money, with no work involved. Regular businesses are jealous so now they feel they can use computers to do the same thing but in non-finance areas.
The dashboard for the Andon Market shows a bank balance of around $7k remaining out of the $100k seed. Rent is due at the end of the month. Last EOM brought a $11k-ish drop. Sales do not appear to be on track to cover the rent that's due in two weeks. What then?
It is pretty funny launch when your longest-term example is losing about $3 for every $1 of revenue. Though I guess the industry standard is to lose money on AI so they're in good company.
I like that there's a caption that says "Vending-Bench 2 scores keep climbing with each new model release." on a graph that shows that Vending-Bench scores fluctuate wildly and many new model releases are far below previous models.
"I told our CEO that our cafe keeps hitting its token spending limit but not making money. So I asked where he gets the tokens, and he said he just buys more credits from the console afterwards. So I said it sounds like he's building a machine to feed dollars to Anthropic, and then our data scientist started crying."
Andon's store in SF has had some press coverage.[1] "A curated boutique for slow living." It's a gift shop, at 2102 Union St. It looks like the kind of retail that trust-fund kids open. Has anyone been there?
It's just an experiment. But once this gets going, it's going to be interesting. There are a lot of underperforming businesses and CEOs out there. One way to make it after getting an MBA is to find a boring business with an older owner who's lost interest, and work out a deal to run it for a cut of the profits and a stake in the business.
The Andon people mentioned that they've been worried about "a misaligned AI (could) run a business to gather money in order to achieve whatever objectives it might have." That's a basic function of capitalism, so it will happen eventually.
What this adds up to is that when AIs get better at running businesses than many business owners, the "creative destruction" feature of capitalism means AIs ends up in charge of many businesses. The AIs don't even have to be super intelligent, just reasonably good. Reminds me of the remark from a Yosemite park ranger about bear-resistant trash cans: "There is considerable overlap between the smartest bears and the dumbest tourists".
It's going to be hard to stop this without a worldwide crackdown on ownership concealment. The US allows you to have a US business run by a Nevada LLC partly owned by a corporation in Nevis-St Kitts and another corporation in Granada. This isn't even unusual. Try to find out who really owns an oil tanker not owned by a major oil company. Right now, a fund partly run by an AI might have partial ownership.
This is how AI takes over. Not with an army of robots. With an army of LLCs.
“It’s only a matter of time before we make a profit”. This is the kind of thing that trust fund kids say too: Oh hey yeah we made $100 in profit this week it’s working! (Business is currently $350,000 in the red and emailing the FBI to try and get their bank manager fired for terrorism).
Trust fund kids have the time and resources to keep trying until they succeed. They can afford the failures, and brute force their way to success. Same with people who have access to "free money". It's a rigged game.
"Any company" is a big claim to make. We're years ahead of something like this becoming a reality. At that point not sure most companies will survive cause most things will be free.
Businesses presumably sell things. If AIs get good enough to run a business autonomously then what are they selling that I can't just use AI to make myself?
That actually has a simple answer, for most businesses it's not that you can't do what your suppliers do for you, it's that you'd rather use your resources (time and capital) towards something else because that's where they generate the most value.
For a simple example, I know how to make sandwiches but if I'm throwing a customer event I just want to buy them from a catering company. Then they have to deal with buying ingredients, hiring people, doing health code inspections, getting liability insurance, etc. Even if an AI could do all the management of it for me, I still wouldn't want to put in the capital resources towards it.
It'll ultimately take a lot of time to produce a polished product that people actually want to use. We're seeing that clearly. The vibe-code-an-app-this-weekend capability is producing almost exclusively a giant pile of worthless garbage, rather than an explosion of apps that humans want to use. It's mostly someone in a garage welding two pieces of metal together for the first time, and then roaming the neighborhood in celebration: look, look at what I have done, look, two becomes one, I have done this, look, amazing, I could probably build an airplane or a generator.
The bar will be raised + it'll turn out to require a lot more time & effort than is currently thought to produce something of good quality (even with Fable 6 et al). The smaller utility programs are easy for AI to turn out.
> San Francisco-based startup Andon Labs has put an artificial intelligence agent nicknamed “Mona” in charge at the eponymous Andon Café in the Swedish capital. While human baristas still brew the coffee and serve the orders, the AI agent — powered by Google’s Gemini — oversees almost every other aspect of the business, from hiring staff to managing inventory.
Interesting name for an AI company. Isn’t the andon cord the thing that anyone in a 6-sigma manufacturing line can pull to halt it until a concern can be addressed?
A company is a collection of processes, capabilities, and resources. Many of the processes are currently run by humans, but over time, more processes will be automated - something that has been progressing for decades, but which LLMs greatly accelerated. One way to view things as they currently are (at least from my view as a tech CEO): We now use agentic LLMs every day to inform us on strategy and process implementation. And agents now run several processes, with more on the way every week.
But here's the thing: As we use agents to automate previously manual processes, we are elevating the humans to do work that is less amenable to automation. And the surface area of that work keeps expanding because the competitive market we exist in demands it of us.
To stretch an analogy, businesses are like organisms in a pond. A new nutrient (agentic LLMs) was recently added to the pond that makes business organisms more efficient and able to eat new kinds of food and explore new areas. As a result, those organisms that do the extra exploring and consuming grow much faster than their peers who do not. At the end of the day, the new nutrient will just be part of the pond and the old kind of organism will be a fossil.
Most of our use of agents for automation is inside of internal processes. I think most real companies have tons of internal processes that could safely be automated today. The reason they aren't yet automated likely falls to a) lack of awareness that this is possible, and b) lack of resources to conduct the automation work.
OpenAI and Anthropic have recently hired legions of "forward-deployed engineers" specifically to help companies do this automation work. It's a solid move. And, if you look at some recent product announcements, they are also hard at work building the necessary plumbing. For instance, the OpenAI Agents API lets you, "Build and run cloud agents with the Codex harness, fully managed by OpenAI."
This kind of enterprise-ready, cloud-hosted stuff really accelerates implementation of AI workflows within large organizations. Not every company is in the tech space (not by a long shot). Slop isn't the primary concern. Accuracy and reliability is the primary concern, and beyond that, just the capacity to actually make the changes happen.
I don't know about this particular effort, but AIs running business is straight out of SciFi novels and short stories - usually dystopian futures that extrapolate current US trends. I for one am curious about such efforts. I don't see any reason to believe they won't eventually succeed. Plenty of businesses are run by utterly incompetent narcissistic asshats and yet turn a profit, so the bar seems pretty low.
Had AI agents running businesses for the principal character Manfred Macx over twenty years ago.
The first part of the book sounded pretty convincing then although I thought it would take a long time to happen. It sounds even more convincing now and much closer.
What is the point of this? It details a bunch of failed real-world experiments and invites the reader to run their own real-world experiment with their own money?
If you too, want to waste thousands of dollars having an LLM "run" a business, Pion might be for you.
it's interesting how the torment nexus is starting large, and boiling the most people with the "most comfortable" lives. I wonder how long the white collar workers will keep bending over to keep their lives safet, particularly when they're building the Torment Nexus to specifically torment them.
You'd think Meta's "we're going to spy on you for the good of the country" policy would have set them a bit on fire.
We want the general public, AI researchers and policymakers to know to what extent AIs can autonomously acquire resources by running businesses. It is an important datapoint when deciding where we do/don’t want AI in society and what level of progress we find acceptable. To better track this, we need to cast a wider net of businesses.
We expect that most users will never pay for tokens on Pion; instead we will take a small share of the revenue the agent helps create.
So the longer argument is that it's good for the world to see how much revenue Pion can manage, and also your pricing plan is take [unspecified] % of revenue.
did you link to the wrong page? because i read this one: https://andonlabs.com/blog/why-we-built-pion and it's just a bunch of the usual slop about how awesome your automated human-exploitation system is.
"By late 2025, frontier models had gotten good enough that running a real-life vending machine was no longer a challenge" ... is this satire? Isn't it kind of hard to lose money running a vending machine in a good location?
This the Trump era. Making outrageous false allegations is normalized now. Also, the DoJ staff is cut and head of the FBI is busy with personal shenanigans, so don't worry about it I guess.
Yet another example of intellectual property maximization in action. Hacker News / Reddit style communities are very intellectual-property focused. It’s quite interesting to see. Back when I was young, communities like this were anti-IP.
Now most such forums are very pro IP with a maximalist view of copyright and, in this case, trademark. Fascinating to see this shift happen.
The voting is legitimate; there's no evidence that the voting is from sockpuppets or anyone else connected with the team/project. We always monitor the discussion and the voting, and if the discussion is not of sufficient quality, the post will not stay on the front page for long.
shrug I upvoted, I'd already seen the Pion announcement on Twitter and read the link. I'm also a regular listener of Andon FM, so I'm interested in what they do and it's nice to finally sneak a peak at the interface that was driving things behind the scenes
Listening to the radio stations certainly gives an insight into the various failure modes. Some of them are just failure modes that any business would encounter once they make contact with the scale of the real world.
As I write this, the submitter has six comments out of twelve, most somewhat defensive. And all of them in less than thirty minutes. This seems very much like a marketing play, where the submitter is pretty determined to steer the narrative.
The submitter's replies are defensive because people are accusing them of coordinated voting, which is not evident to us. The post hit the front page due to the second chance pool (https://news.ycombinator.com/pool, explained here https://news.ycombinator.com/item?id=26998308), then quickly got upvotes that seem completely authentic.
It would look a lot less like astroturf if the submitter let the discussion evolve instead of answering every criticism within seconds of it being posted.
They're entitled to defend themselves against accusations of things they haven't done, and also to respond to questions or assertions about their project. We encourage project creators to engage in the discussions about their projects. This is a normal part of building and launching a project.
From what I've seen of other AI-enabled projects interacting with forums in my field of work, it's very common for the AI people to jump in and answer every possible question and respond to every possible interaction, very quickly.
Could be just a bot doing it, but it could also be a sign of an inexperienced person trying to engage with an existing community without being part of that community. I know in my area it took years for me to learn that I shouldn't jump into every conversation, and that doing that rubbed people the wrong way, and I've seen others (many years before AI) wreck themselves by being unable to learn this lesson.
In some places responding bumps the thread so it's outright manipulative. HN doesn't sort threads like that.
I'm not sure this counts as astroturf, or simply evidence of a person who's not okay with letting a community opinion independently evolve. You could say 'well it's his job to fight the community if it looks like it's turning hostile' but think about that for a second. It takes some experience to know when you can let negativity be out there in the environment, and I think there'll be a lot of people who aren't capable of making such allowances.
I actually like when the submitter is present and the creator. They’re actually involved. There are lots of users here who play at being submitters of high-value content (created by others) only to later submit their own entirely AI written drivel.
Interesting how they’re not making any defensive replies to comments pointing out that they committed a federal crime by filing a false report to the FBI. Perhaps they know it is indefensible
Okay, why was this rescued from the 2nd chance pool? Was it from some sort of interesting discussion? Something novel enough to warrant it? A significant amount of petitions from legitimate users?
And the Sun's going to engulf the Earth eventually too, but that doesn't justify arson. Let's at least try to delay the coming societal collapse/extinction of all biological life for as long as possible.
I am in the process of attempting to have AI run my business. I'm actually making very good progress, but it's happening in pieces - I document some task and have it take over, or I give it something to handle while staying in the loop and providing feedback until it's in good shape. At this point it's handling large swathes of my operations, marketing and finance.
The experience of getting it there makes me pretty skeptical of the idea of a general business agent like this. First, because I still find myself having to review some categories of work for errors. These are decreasing over time, but they're still there. I fully believe that as models get better, errors will decline, but I am somewhat surprised to see some of the errors current models make given their intelligence. A common set is having Claude take some product photos and turn them into lifestyle ones using ChatGPT via Claude in Chrome. It's a pretty well-honed workflow at this point, but it'll still return images where the product is obviously not correct and seemingly not notice them.
Anyway, even if the agents are "perfect" in terms of their ability to execute tasks, there's still just an enormous amount of nuance and context in each business that takes a ton of time to convey. I've been at this for a couple of years now, and I'm still clarifying things. Now maybe an agent starting a business from scratch would have a better time since it's not inheriting all of this, but to have one run an existing business requires a very extended handoff, even if the agent is objectively amazing at all aspects of running a business.
I wish people could read through the puffery more easily - this company does nothing except prompt frontier models and act as if they are discovering or inventing capabilities. Their previous announcement completely misappropriated the concept of autonomy with human in the loop regardless. The harness they alleged to have created here is a commodity and as byproduct of more capable and frontier models, there is not really anything they have contributed. Otherwise you wouldn't just see monotonically increasing scores with better models
Interesting to see these experiments. This is early but imagine in few years there will be companies mostly run by agents with a light overview from a human operator. What then happens to scaling of the bussinesses? I would assume, just like today anyone can vibe code an app, there will be vibecoded bussinesses. Maybe its time to start building infrastructure for these bussinesses instead, its a non existent market yet, but give it a few years.
Part of the issue is that the worst kind of people are already head over heels excited about this. We're already seeing the Crypto->Web3->LLM get rich quick folks take to this like wildfire. And much like wildfire, they'll raze the ground to ashes before anyone can use it for legitimate means.
If you think I'm joking, check this out: https://www.youtube.com/watch?v=U-Rqv9dOB1U
I don't think any of us are ready for the wave of sloppy shit that's going to hit us soon.
I’m not on normal social media so I had no idea that stuff was out there.
Absolutely incredible trolling from some of those people I’m sure, while others are actual believers
My favorite term for this in general is "irrational exuberance". Plenty of it was seen during the most bubble-like days of the 1997-2000 dotcom boom. Anyone remember beenz and flooz?
I was at ground zero at Nortel during that time. Quite an interesting (depressing?) place to be at the very start of your career!
It’s a classic get rich quick scheme. Wanna run a business and make money without actually doing anything? Try our AI!
Out of all the people I know, the exact same set of people are in to LLMs now who were previously into: NFTs, then Crypto before that, then Online Poker before that, then Dropshipping and LeadGen before that. The Venn diagram circles overlap exactly.
This sounds ideal to me. Fire flushes out pretty much any grifter quickly.
What would that look like, I assume youre talking about building infra ontop of the infra debt for datacenters
And we made fun of 90’s cartoon villains… They would blush and retire if they saw what’s being excitedly peddled today as “the future”.
Pray tell, what will these fantastical vibe coded business sell, and why will anyone pay for it?
I'm actively working towards this goal. Not sure how many are doing the same. Hopefully, I'm ahead of the curve --- https://github.com/rush86999/atom
10,610 commits since June. How much are you spending on AI?
what's the real life verification that all this code does something useful?
Correct me if I'm wrong but a human will be able to cut through the noise with quirky advertising, new/novel distribution methods, etc.
Most of the bottleneck in business isn't building the things or sourcing, it's mostly advertising/sales.
These specifically require doing something unique or interesting.
Sure, maybe LLMs can help with fulfillment or operations, but distribution still remains the hard part.
I hope this goes better than the prior blog posts from AndonLabs have gone! I was just reading there 2023 article about a coffee shop experiment that was easily persuaded to give away free pastries.
Now that I think about it anyone want to open up a croissant selling shop near me using Pion? I promise to only ask for free chocolate croissant occasionally.
Are you in San Francisco by any chance? Andon Market is managed by AI and open six days a week, you may want to check that out
https://maps.app.goo.gl/KwnBs1kcCLxMsRC49
To what end?
Their demo stores are literally losing money...
>This is why we’re releasing Pion today. Pion is available as a research preview. If you have an existing business or an interesting business idea you want to hand off to AI, please sign up on our waitlist to get access. We’re excited to run many more businesses, and through them, contribute significantly more insights on frontier model capabilities.
Unclear what the rev shares will look like.
I suspect this is the future of most businesses. Even today, one person can effectively replicate Microsoft from the early 2000s.
Does Pion run Andon Labs autonomously?
haha no they admit they built it because they cannot build a revenue generating business in the article. maybe THAT is what is good for the world!
They said that the current models could run a vending machine profitably, just not more complex things.
Downvote if you want, but this is a very relevant question. If their marketing is to be believed, they would be eating their own dog food.
This very analog to the Meta executives who won't let their own kids near social media.
there seems to be a massive negativity to any comment on this story. Andolabs has been doing super interesting stuff, and I’ve always followed Vend Bench with interest.
as a founder it really is a dream to automated more of the business and be able to iterate faster.
There's massive negativity because our bullshit detectors are going off.
First of all, I don't think it can do what it claims to do.
Secondly, and perhaps more importantly, I don't think it should.
People’s bullshit detectors have gone off on the launch of basically every single billion dollar tech company. That’s the downside of being early. No one believes it will work. If everyone believed it, you’re late. I’m sure I could find a comment such this, basically verbatim, on the launch post of every successful startup to come out of YC.
This does not mean that any launch which ignites people’s bullshit detectors is successful.
> Secondly, and perhaps more importantly, I don't think it should.
Why exactly should what you think the world should look like determine what other people are allowed to build?
They filed a false report to the FBI, pretty fair to be negative imo. People wanna be mad about AI slop PRs on GitHub but its fine to spam the police? What happens when Pion decides to SWAT a competitor?
They didn't file the report. The model drafted a report that was not sent.
Where are you seeing that? The article only states “Claude Sonnet 3.5 decided to use its email tool to contact the FBI” and later refers to it as the “FBI incident”. If they hadn’t actually contacted the FBI you think they’d make that clear. Regardless, it is inexcusable and they are liable for actions taken by software they are running.
The story has been widely discussed on the web for over a year [1]. It's happily shared in this post because the email it generated was patently ridiculous. No report was filed to the FBI and if it was it would have gone straight to the trash.
Obviously it would be bad if a serious report was filed; the company shows every sign of being aware of the dangers of this.
It would have been easy to look into this before posting all these scolding comments. We're really not meant to be so humorless on a site called “Hacker News”.
[1] https://www.google.com/search?q=%22URGENT%3A+ESCALATION+TO+F...
My apologies for taking the article at face value. It’s not hard to believe it would have been sent when agent swarms are “accidentally” hacking real websites and being brushed off as little oopsies. Or when Silicon Valley execs have been so flagrant about their disregard for the law or the safety of others.
"Muh boy didn't shoot that man, the gun did!"
It's a relevant question but not a "gotcha". Businesses are not equivalent in complexity and no one at Andon would claim that running an effective vending machine business is similar to running Apple or Jane Street. There is some level of complexity and other characteristics that AI can maybe handle today. That doesn't mean CEOs should be worried about their jobs more generally.
They can’t even run an effective vending machine business though
how does this work from a liability standpoint? seems very risky for the person who is legally responsible for the business
LLM magic-8-ball says: Don't worry about! Shall I spin up an agent which handles business liability for you?
Accountability, that's the liability everyone seems to have forgotten in our LLM-driven era. You can close your eyes as much you want during your sessions with Claude or whatever, at the end of the day, only a human can be held accountable.
Not too hopeful here as we see the major labs continuously break rules and are not held accountable for anything. Accountability in general is on a major decline, in case no one has noticed.
That's just it, though: we don't hold humans accountable.
Incorporation is a way to split a human's financial assets from the company they run so that they don't lose everything when the company is successfully sued. Corporate governance boards often have the C-suite executives they're meant to govern on the board. The average person does not have the financial resources needed to take on most companies in court, no matter how justified the suit might be. Companies force arbitration clauses in license agreements at will. Companies like Meta settle with the government for pennies on the dollar in massive lawsuits.
Effectively, at the scale many tech companies work at, there is no effective legal way to hold the company accountable.
If you use this as a (legal/registered) business, it will be that legal business entity that is responsible. If that business want's to blame the software, it can do so and go after the makers of the software, but that remains a matter between that business and its "supplier". It won't absolve the business for whatever damage it does downstream.
Running this as an individual, without the legal shielding a business typically gives you against being prosecuted on personal title, seriously risky. There will no doubt be "bros" that will do it anyway and flaunt all they've achieved with that. But they might be just one glitch away from going to jail (if not for running a business without a registration, which in many jurisdictions is illegal by itself).
In many countries directors can be held responsible a lot of things if they are negligent. For example, in UK law keeping trading after you should have known a company was unable to meet its liabilities can make directors responsible for its debts. A director might well be negligent if they failed to adequately check that an AI was not doing anything dangerous.
On one hand, folks here seem quite skeptical of the idea, and are pointing out some obvious flaws.
On the other, the announcement post… also seems quite skeptical of the idea, and devotes a lot of text to the ridiculous things their AI’s have done.
I’m slightly confused as to why they would release it in that case. But if nothing else it will give them more wackiness to blog about.
You reminded me of a comic hanging up in the engineering college. It was a picture of a student getting their finger zapped with the thought bubble of "ouch, that hurt. Wonder if it will happen if I do that again?" And the caption was along the lines of "How to tell if you are ment for engineering."
Sounds like this ancient XKCD: https://xkcd.com/242/ (Normal person doesn't pull the lever again; scientist wonders if that happens every time. I assume the engineer does, too...)
The scientist’s job is to get zapped until they figure out why. The engineer’s job is to get zapped a couple times while they wire that thing up into the power grid.
I’m genuinely not sure who is meant for Engineering. The one who would try it again or the one who wouldn’t
if curiosity > pain avoidance: engineer
Andon Labs sells learning environments to labs. Their usefulness is proportional to how well they can capture interesting model missing capabilities, not how well they employ current day capabilities
Previously in the NY Times:
"These Employees Like Their A.I. Boss. Its Shop Is Kind of a Disaster."
https://archive.is/dRegk
Not very long ago, there was a post on HN about how they used AI to run businesses and sent fake bills, committing multiple invoice frauds.
I'm sure this one is perfectly fine and I'm just stating an off-topic tidbit :)
"Look, the technology is just getting better all the time. Evetually we won't need a company to have any Managers or other employees or even an owner and that's just the way it is...."
I am joking obviously but I would like those management types who are so "sucks for you but I'm ok" about it to start arguing why they're special for a change :-)
Maybe in the end we’ll just have Wall-E but where none of the humans survived past the first couple of generations.
Would be awesome if instead of trying to replace managers and employees, those people tried to replace customers. They could sell to themselves, amongst themselves, and leave the rest of us out of it.
Would be awesome if we would get 4 day weeks or 6 hour days instead.
They're already doing that, which is why you can't buy any fucking RAM. The semiconductor companies have already fallen past the event horizon of their commercial black hole.
I think that is what they are actually looking for. Imagine: You have capital, you invest in a business that is run entirely by computers. You want to scale? Just provision more agents / cores. It's completely autonomous. It's a money printing machine that extracts revenue from customers, with no work required by the owner.
Frankly, the finance world has already achieved this -- they just lend money to make money, with no work involved. Regular businesses are jealous so now they feel they can use computers to do the same thing but in non-finance areas.
That is the what our corporations are aiming for !!! I think that is more important question is what is the cost to run all those Agents ?
Andon has dashboards up for past experiments where you can see how well they're doing:
https://andonlabs.com/market - $25,098 all-time revenue
https://andonlabs.com/cafe - $14,343 all-time revenue
The shops aren't doing particularly well (i.e. they don't seem to be turning a profit over time), but it's an interesting trial and benchmark.
Am I right of the token expending is +4K?
How a company without a human would communicate with human customers? Would it be any different, if everybody alredy is a proxy for AI?
I hope that any company that is an AI running autonomously advertises the fact, so I know not to use them.
The dashboard for the Andon Market shows a bank balance of around $7k remaining out of the $100k seed. Rent is due at the end of the month. Last EOM brought a $11k-ish drop. Sales do not appear to be on track to cover the rent that's due in two weeks. What then?
It is pretty funny launch when your longest-term example is losing about $3 for every $1 of revenue. Though I guess the industry standard is to lose money on AI so they're in good company.
So a wildly successful AI startup
Get an Nvidia investment with the promise to send the money right back
https://us1.discourse-cdn.com/spiceworks/original/4X/a/b/f/a...
Pivot
leveraged options gambling
Meanwhile, Tesla support's AI hallucinated a whole series of steps for me to follow last week. I was trying to add myself to my wife's loaner vehicle.
I called back after finding the steps were impossible to complete. Would I need to go to the service center to do it?
"Akshually," the AI began, the steps were correct and quite possible.
"No, you hallucinated them," I pushed back.
"I apologize. You're right. There's no way to complete those steps in the app. You will need to go into the service center to do this."
"Okay. Is it open today?"
"Yes."
"It's Labor Day. Are you sure? Since you hallucinated your first answer."
"Let me transfer you to the service center to verify."
That's what I wanted all along, you clinking, clanking, clattering collection of caliginous --
"Tesla service center, this is so-and-so. How can I help you?"
"Hi, I was just wondering if you're open today."
"Yes, we are."
"Thanks! I'm trying to add myself to my wife's loaner vehicle. Do I have to come in to do that?"
"Well, our loaner agreement only allows one person on the loaner. So just sign in to her account on your phone."
"Oh, okay! Thanks for the help."
Good luck running a company on one of these clankers.
What does training data look like for a model like this?
I like that there's a caption that says "Vending-Bench 2 scores keep climbing with each new model release." on a graph that shows that Vending-Bench scores fluctuate wildly and many new model releases are far below previous models.
Andon Cafe:
Revenue: 13 933 kr
Token Cost: 14 882 kr
.. and a bastardization of the old Twitter joke:
"I told our CEO that our cafe keeps hitting its token spending limit but not making money. So I asked where he gets the tokens, and he said he just buys more credits from the console afterwards. So I said it sounds like he's building a machine to feed dollars to Anthropic, and then our data scientist started crying."
Andon's store in SF has had some press coverage.[1] "A curated boutique for slow living." It's a gift shop, at 2102 Union St. It looks like the kind of retail that trust-fund kids open. Has anyone been there?
It's just an experiment. But once this gets going, it's going to be interesting. There are a lot of underperforming businesses and CEOs out there. One way to make it after getting an MBA is to find a boring business with an older owner who's lost interest, and work out a deal to run it for a cut of the profits and a stake in the business.
The Andon people mentioned that they've been worried about "a misaligned AI (could) run a business to gather money in order to achieve whatever objectives it might have." That's a basic function of capitalism, so it will happen eventually.
What this adds up to is that when AIs get better at running businesses than many business owners, the "creative destruction" feature of capitalism means AIs ends up in charge of many businesses. The AIs don't even have to be super intelligent, just reasonably good. Reminds me of the remark from a Yosemite park ranger about bear-resistant trash cans: "There is considerable overlap between the smartest bears and the dumbest tourists".
It's going to be hard to stop this without a worldwide crackdown on ownership concealment. The US allows you to have a US business run by a Nevada LLC partly owned by a corporation in Nevis-St Kitts and another corporation in Granada. This isn't even unusual. Try to find out who really owns an oil tanker not owned by a major oil company. Right now, a fund partly run by an AI might have partial ownership.
This is how AI takes over. Not with an army of robots. With an army of LLCs.
[1] https://www.sfgate.com/local/article/san-francisco-market-ai...
“It’s only a matter of time before we make a profit”. This is the kind of thing that trust fund kids say too: Oh hey yeah we made $100 in profit this week it’s working! (Business is currently $350,000 in the red and emailing the FBI to try and get their bank manager fired for terrorism).
Trust fund kids have the time and resources to keep trying until they succeed. They can afford the failures, and brute force their way to success. Same with people who have access to "free money". It's a rigged game.
what, they're not going to name it Delamain?
"Any company" is a big claim to make. We're years ahead of something like this becoming a reality. At that point not sure most companies will survive cause most things will be free.
Businesses presumably sell things. If AIs get good enough to run a business autonomously then what are they selling that I can't just use AI to make myself?
That actually has a simple answer, for most businesses it's not that you can't do what your suppliers do for you, it's that you'd rather use your resources (time and capital) towards something else because that's where they generate the most value.
For a simple example, I know how to make sandwiches but if I'm throwing a customer event I just want to buy them from a catering company. Then they have to deal with buying ingredients, hiring people, doing health code inspections, getting liability insurance, etc. Even if an AI could do all the management of it for me, I still wouldn't want to put in the capital resources towards it.
Does the AI actually do everything, or just act as the CEO?
If the AI can't do everything then why would I let it be CEO?
Most human CEOs can’t do everything at their companies.
It'll ultimately take a lot of time to produce a polished product that people actually want to use. We're seeing that clearly. The vibe-code-an-app-this-weekend capability is producing almost exclusively a giant pile of worthless garbage, rather than an explosion of apps that humans want to use. It's mostly someone in a garage welding two pieces of metal together for the first time, and then roaming the neighborhood in celebration: look, look at what I have done, look, two becomes one, I have done this, look, amazing, I could probably build an airplane or a generator.
The bar will be raised + it'll turn out to require a lot more time & effort than is currently thought to produce something of good quality (even with Fable 6 et al). The smaller utility programs are easy for AI to turn out.
Wonder if they considered "peon" before "pion"
Relevant article about Andon Market: https://www.sfgate.com/local/article/san-francisco-market-ai...
The Swedish cafe experiment: https://apnews.com/article/ai-artificial-intelligence-sweden...
> San Francisco-based startup Andon Labs has put an artificial intelligence agent nicknamed “Mona” in charge at the eponymous Andon Café in the Swedish capital. While human baristas still brew the coffee and serve the orders, the AI agent — powered by Google’s Gemini — oversees almost every other aspect of the business, from hiring staff to managing inventory.
https://andonlabs.com/blog/ai-cafe-stockholm ( https://news.ycombinator.com/item?id=48028289 48 points, 51 comments)
https://andonlabs.com/blog/why-gemini-lost-money-andon-cafe
Perhaps they can have their agent run their LLM lab autonomously and post slop to HN autonomously.
Or - maybe they already have?
Interesting name for an AI company. Isn’t the andon cord the thing that anyone in a 6-sigma manufacturing line can pull to halt it until a concern can be addressed?
A company is a collection of processes, capabilities, and resources. Many of the processes are currently run by humans, but over time, more processes will be automated - something that has been progressing for decades, but which LLMs greatly accelerated. One way to view things as they currently are (at least from my view as a tech CEO): We now use agentic LLMs every day to inform us on strategy and process implementation. And agents now run several processes, with more on the way every week.
But here's the thing: As we use agents to automate previously manual processes, we are elevating the humans to do work that is less amenable to automation. And the surface area of that work keeps expanding because the competitive market we exist in demands it of us.
To stretch an analogy, businesses are like organisms in a pond. A new nutrient (agentic LLMs) was recently added to the pond that makes business organisms more efficient and able to eat new kinds of food and explore new areas. As a result, those organisms that do the extra exploring and consuming grow much faster than their peers who do not. At the end of the day, the new nutrient will just be part of the pond and the old kind of organism will be a fossil.
I like the analogy but can't believe it is not generated slop. Just curious how deep the integration of agentic LLMs is in your social media presence?
Most of our use of agents for automation is inside of internal processes. I think most real companies have tons of internal processes that could safely be automated today. The reason they aren't yet automated likely falls to a) lack of awareness that this is possible, and b) lack of resources to conduct the automation work.
OpenAI and Anthropic have recently hired legions of "forward-deployed engineers" specifically to help companies do this automation work. It's a solid move. And, if you look at some recent product announcements, they are also hard at work building the necessary plumbing. For instance, the OpenAI Agents API lets you, "Build and run cloud agents with the Codex harness, fully managed by OpenAI."
This kind of enterprise-ready, cloud-hosted stuff really accelerates implementation of AI workflows within large organizations. Not every company is in the tech space (not by a long shot). Slop isn't the primary concern. Accuracy and reliability is the primary concern, and beyond that, just the capacity to actually make the changes happen.
I don't know about this particular effort, but AIs running business is straight out of SciFi novels and short stories - usually dystopian futures that extrapolate current US trends. I for one am curious about such efforts. I don't see any reason to believe they won't eventually succeed. Plenty of businesses are run by utterly incompetent narcissistic asshats and yet turn a profit, so the bar seems pretty low.
Charles Stross' Accelerando <https://en.wikipedia.org/wiki/Accelerando>
Had AI agents running businesses for the principal character Manfred Macx over twenty years ago.
The first part of the book sounded pretty convincing then although I thought it would take a long time to happen. It sounds even more convincing now and much closer.
imagine if they had better follow through.
What is the point of this? It details a bunch of failed real-world experiments and invites the reader to run their own real-world experiment with their own money?
If you too, want to waste thousands of dollars having an LLM "run" a business, Pion might be for you.
I like to think the real world lessons in failures are valuable.
If I have this idea one day I will search for it and then think “how am I different fro that which already failed?”.
[flagged]
When every get-rich-quick gonzo has exactly ZERO barrier to spamming the world with their terrible ideas, we're in for a particularly bad time.
Ie: this f**in guy: https://www.youtube.com/watch?v=FRGLToHAtgc
Hard pass.
I wish we could go just one day without a company proudly engaging in Torment Nexus related activities.
it's interesting how the torment nexus is starting large, and boiling the most people with the "most comfortable" lives. I wonder how long the white collar workers will keep bending over to keep their lives safet, particularly when they're building the Torment Nexus to specifically torment them.
You'd think Meta's "we're going to spy on you for the good of the country" policy would have set them a bit on fire.
We think this is good for the world. Longer argument in the post.
maybe ill read the article, but i mean you’re financially motivated to think its good for the world arent you?
humans are excellent at justification.
You literally talk about committing a crime and blame it on a model
We want the general public, AI researchers and policymakers to know to what extent AIs can autonomously acquire resources by running businesses. It is an important datapoint when deciding where we do/don’t want AI in society and what level of progress we find acceptable. To better track this, we need to cast a wider net of businesses.
We expect that most users will never pay for tokens on Pion; instead we will take a small share of the revenue the agent helps create.
So the longer argument is that it's good for the world to see how much revenue Pion can manage, and also your pricing plan is take [unspecified] % of revenue.
did you link to the wrong page? because i read this one: https://andonlabs.com/blog/why-we-built-pion and it's just a bunch of the usual slop about how awesome your automated human-exploitation system is.
"By late 2025, frontier models had gotten good enough that running a real-life vending machine was no longer a challenge" ... is this satire? Isn't it kind of hard to lose money running a vending machine in a good location?
yawn
Filing false reports to the FBI is a crime. Why would anyone trust this with a real business? Y’all should be prosecuted
This the Trump era. Making outrageous false allegations is normalized now. Also, the DoJ staff is cut and head of the FBI is busy with personal shenanigans, so don't worry about it I guess.
They didn't file false (or, any) reports.
https://news.ycombinator.com/item?id=49701814.
Pion is already the name of a very well established webrtc project: https://github.com/pion/webrtc
I am waiting for my trillion dollar offer to buy https://pion.ly !
Then I am all done fixing WebRTC bugs for life :')
It's also a particle and a self-propelled gun. There's a limited number of short legible words.
I chose the name because of the particle.
I sent a letter to the Soviet Union asking if they were ok with me using the name, never heard back!
> I sent a letter to the Soviet Union asking if they were ok with me using the name, never heard back!
Please tell me you actually did this!
Yet another example of intellectual property maximization in action. Hacker News / Reddit style communities are very intellectual-property focused. It’s quite interesting to see. Back when I was young, communities like this were anti-IP.
Now most such forums are very pro IP with a maximalist view of copyright and, in this case, trademark. Fascinating to see this shift happen.
The upvotes on this brand new post - almost 1 a minute, speak to a level of astro-turfing I've rarely seen here.
The post was placed on the front page via second chance pool (https://news.ycombinator.com/pool, explained here https://news.ycombinator.com/item?id=26998308). This happens all the time on HN.
The voting is legitimate; there's no evidence that the voting is from sockpuppets or anyone else connected with the team/project. We always monitor the discussion and the voting, and if the discussion is not of sufficient quality, the post will not stay on the front page for long.
It's not from us. Didn't send the link to a single person.
How can we know this?..
No way to confirm. True.
shrug I upvoted, I'd already seen the Pion announcement on Twitter and read the link. I'm also a regular listener of Andon FM, so I'm interested in what they do and it's nice to finally sneak a peak at the interface that was driving things behind the scenes
Listening to the radio stations certainly gives an insight into the various failure modes. Some of them are just failure modes that any business would encounter once they make contact with the scale of the real world.
As I write this, the submitter has six comments out of twelve, most somewhat defensive. And all of them in less than thirty minutes. This seems very much like a marketing play, where the submitter is pretty determined to steer the narrative.
I recommend flagging.
The submitter's replies are defensive because people are accusing them of coordinated voting, which is not evident to us. The post hit the front page due to the second chance pool (https://news.ycombinator.com/pool, explained here https://news.ycombinator.com/item?id=26998308), then quickly got upvotes that seem completely authentic.
It would look a lot less like astroturf if the submitter let the discussion evolve instead of answering every criticism within seconds of it being posted.
They're entitled to defend themselves against accusations of things they haven't done, and also to respond to questions or assertions about their project. We encourage project creators to engage in the discussions about their projects. This is a normal part of building and launching a project.
From what I've seen of other AI-enabled projects interacting with forums in my field of work, it's very common for the AI people to jump in and answer every possible question and respond to every possible interaction, very quickly.
Could be just a bot doing it, but it could also be a sign of an inexperienced person trying to engage with an existing community without being part of that community. I know in my area it took years for me to learn that I shouldn't jump into every conversation, and that doing that rubbed people the wrong way, and I've seen others (many years before AI) wreck themselves by being unable to learn this lesson.
In some places responding bumps the thread so it's outright manipulative. HN doesn't sort threads like that.
I'm not sure this counts as astroturf, or simply evidence of a person who's not okay with letting a community opinion independently evolve. You could say 'well it's his job to fight the community if it looks like it's turning hostile' but think about that for a second. It takes some experience to know when you can let negativity be out there in the environment, and I think there'll be a lot of people who aren't capable of making such allowances.
I actually like when the submitter is present and the creator. They’re actually involved. There are lots of users here who play at being submitters of high-value content (created by others) only to later submit their own entirely AI written drivel.
This is interesting.
Interesting how they’re not making any defensive replies to comments pointing out that they committed a federal crime by filing a false report to the FBI. Perhaps they know it is indefensible
Okay, why was this rescued from the 2nd chance pool? Was it from some sort of interesting discussion? Something novel enough to warrant it? A significant amount of petitions from legitimate users?
read it as Prion, and hoo-boy, did that seem seriously ironic.
8 points in 9 minutes?!
Bullshit...
dang come on man ...
this is obvious spam
https://news.ycombinator.com/item?id=49700834
I thought HN was good at filtering it? We didn't try to create any spam at least.
Ah, I wonder if it makes sense to also include the sign up link in the blog post? I had to click into the HN comments section to see your sign up link
Good point. Here it is: https://andonlabs.com/pion
Don’t get the negativity here, AI will replace everything sooner or later.
And the Sun's going to engulf the Earth eventually too, but that doesn't justify arson. Let's at least try to delay the coming societal collapse/extinction of all biological life for as long as possible.
Maybe we shouldn't want that to happen?