I should probably confess that as someone who lives in an area with a lot of construction work, I'm also very vulnerable to "prompt injection" when there's a person standing on the middle of the road holding a sign telling me to change course.
I once encountered an intersection with a big "NO ENTRY" sign on the other side. I turned but google maps wouldn't give me another route, so I did a u-turn and came back to it from the side. Which meant I was close enough to read the small text underneath that said "vehicles under 10 tons excepted". I don't think I've ever been so angry at a road sign.
By my work there is a nice clean sign at the main intersection that reads "NO RIGHT ON RED" with a separate smaller crusty looking sign below it that reads "4 to 5 PM" using a much smaller font. Of course the stark difference in signs means everyone just reads the shiny top sign and waits for the green at all times. I keep wanting to modify the sign to highlight the time.
I came across one in Italy that was meant to prevent you from using a street during school days from X to Y am, and Z to W pm, except on weekends, bank holidays and school holidays.
The idea is well-intentioned, but implementing it by making drivers try to parse arbitrarily complex conditionals while driving is unwise.
There's a sign near my house for a school zone with a reduced speed limit, that used to have conditions similar to the GP's example (though not quite as bad) But they recently attached a yellow light to the top of the sign and changed the condition to "when flashing." That's a much more effective solution.
Obviously. But you can also easily look around at the situation and know when the sign is fake and realize it may be a dangerous situation and disobey. Have you ever seen a green sign that says "Proceed" and just run through a red light because of it? No, you see a construction worker, you see big ass trucks, orange signs and warnings of workers everywhere. If you saw oncoming traffic and people in the road, would you just go because the construction worker flipped his STOP sign around?
Also, I thought we were supposed to make autonomous cars better than humans? What's with the constant excusing of the computer because people suck?
They are analysing VLM here, but it's not as if any other neural network architecture wouldn't be vulnerable. We have seen this in classifier models that can be tricked by innocuous-looking objects, we have seen it in LLMs, and we will most likely see it in any end-to-end self-driving model.
If an end-to-end model is used and there is no second, more traditional safety self-driving stack, like the one Mercedes will use in their upcoming Level 2++ driving assistant, then the model can be manipulated essentially without limit. Even a more traditional stack can be vulnerable if not carefully designed. It is realistic to imagine that one printed page stuck on a lamppost could cause the car to reliably crash.
> It is realistic to imagine that one printed page stuck on a lamppost could cause the car to reliably crash.
Realistic, yes. But that'd still be a symptom of architectural issues in the software.
Conceptually the priorities of a car are (in order of decreasing importance) not hitting other moving or stationary objects or people, allowing emergency vehicles to pass unhindered, staying on a drivable surface, behaving predictable enough to prevent other road users crashing, following road signs and traffic laws, and making progress towards the destination (you can argue about the order of the last three). Typically you'd want each of these handled by their own subsytem because each is a fairly specialized task. A system that predicts the walking paths of pedestrians won't be good at finding a route to Starbucks.
The "follow road signs and traffic laws" is easily tricked, like in this article or by drawing road lines with salt. But that should never crash the car, because not hitting anything and staying on the road are higher priority. And tricking those systems is much harder
No! No one in their right mind would even consider using them for guidance and if they are used for OCR (not too my knowledge but could make sense in certain scenarios) then their output would be treated the way you'd treat any untrusted string.
> Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data. Trained and fine-tuned specifically for autonomous driving, EMMA leverages Gemini’s extensive world knowledge to better understand complex scenarios on the road.
You were confidently wrong for judging them to be confidently wrong
> While EMMA shows great promise, we recognize several of its challenges. EMMA's current limitations in processing long-term video sequences restricts its ability to reason about real-time driving scenarios — long-term memory would be crucial in enabling EMMA to anticipate and respond in complex evolving situations...
They're still in the process of researching it, noting in that post implies VLM are actively being used by those companies for anything in production.
I should have taken more care to link a article, but I was trying you link something more clear.
But mind you, everything Waymo does is under research.
So let's look at something newer to see if it's been incorporated
> We will unpack our holistic AI approach, centered around the Waymo Foundation Model, which powers a unified demonstrably safe AI ecosystem that, in turn, drives accelerated, continuous learning and improvement.
> Driving VLM for complex semantic reasoning. This component of our foundation model uses rich camera data and is fine-tuned on Waymo’s driving data and tasks. Trained using Gemini, it leverages Gemini’s extensive world knowledge to better understand rare, novel, and complex semantic scenarios on the road.
> Both encoders feed into Waymo’s World Decoder, which uses these inputs to predict other road users behaviors, produce high-definition maps, generate trajectories for the vehicle, and signals for trajectory validation.
They also go on to explain model distillation. Read the whole thing, it's not long
But you could also read the actual research paper... or any of their papers. All of them in the last year are focused on multimodality and a generalist model for a reason which I think is not hard do figure since they spell it out
This strikes me as a skunworks project to investigate a technology that could be used for autonomous vehicles someday, as well as score some points with Sundar and the Alphabet board who've decreed the company is all-in on Gemini.
Production Waymos use a mix of machine-learning and computer vision (particularly on the perception side) and conventional algorithmic planning. They're not E2E machine-learning at all, they use it as a tool when appropriate. I know because I have a number of friends that have gone to work for Waymo, and some that did compiler/build infrastructure for the cars, and I've browsed through their internal Alphabet job postings as well.
Over the years, the agency has flagged signs that could be confusing. Now, in rules issued last month, it gives states two years to phase out signs that have "obscure" meanings or use pop-culture references that could require drivers "greater time to process." In a statement, the agency said safety is the priority and states "are expected to exercise good judgment."
> In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes.
I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking about how he cut one off at a stop sign. It's weird.
This is a legitimate movement in my eyes. I don’t participate, but I see it as valid. This is reminiscent of the Luddite movement - a badly misunderstood movement of folks who were trying to secure labor rights guarantees in the face of automation and new tools threatening to kill large swaths of the workforce.
The Luddites were employed by textile manufacturers and destroyed machines to get better bargaining power in labor negotiations. They weren't indiscriminately targeting automation, they targeted machines that directly affected their work.
Because life would be so much better if people still had to spin wool and weave cloth by hand, and grow their own food by digging in the earth with no tools.
Use whatever means necessary to stop powerful people from exploiting you and stealing the fruits of your labor. If that struggle involves monkeywrenching their machines, so be it.
But like any tool, the machines themselves can be used for good or evil. Breaking the machines shouldn't be an end in itself.
The 700m people suffering from starvation or malnutrition while we produce excess food would probably rather be digging in the earth with no tools if it meant they got fed.
The Luddites wouldn't have been destroying machines if they had insurance that they would also benefit from the machines, rather than see their livelihoods being destroyed while the boss made more money than ever.
Like the OP, you misunderstand the entire point of the Luddites. Breaking the machines was not an end, it was the tactical means to help illustrate their broader point of how the owning class can arbitrarily ruin their entire lives and livelihoods with absolutely zero recourse or consultation with the impacted people. This is a defining feature of capitalism, and that was their issue.
Your strawman about spinning and digging with no tools is just that, and is irrelevant to the core issue of capitalism.
If the core issue is ending exploitation by capitalists and not about breaking machines, if you don't want to return to a world without automation, if the machine is just a strawman, then why do you describe yourself as "anti-AI" instead of "anti-capitalist" or "anti-exploitation"?
It seems like you identify yourself with the strawman instead of with the core issue.
I am anti-capitalist and exploitation. And I don't think any anti-capitalist person can be pro-AI, not the way it's currently constructed. But people on a startup forum tend to lose their minds if you say you're against either :)
Being anti-AI is not a straw man, it's the logical conclusion of being against exploitation and hierarchical domination. Discussing that nuance here is difficult, to say the least, so it's simpler to say anti-AI.
Unless you're committing serious crimes vandalizing machines to get leverage over a counterparty in a negotiation you're not comparable to the Luddites.
Criminality is an arbitrary benchmark here, cutting people off can be illegal due to the risks involved.
However what’s more interesting is the deeper social contracts involved. Destroying other people’s stuff can be perfectly legal such as fireman breaking car windows when someone parks in front of a fire hydrant. Destroying automation doesn’t qualify for an exception, but it’s not hard to imagine a different culture choosing to favor the workers.
Inflicting damage is usually justified by averting larger damage. Very roughly, breaking a $200 car window is justified in order to save a $100k house from burning down. Stealing someone's car is justified when you need a car to urgently drive someone bleeding to a hospital to save their life (and then you don't claim the car is yours, of course).
I don't think Luddites had an easy justification like this.
I'm pretty sure the Luddites judged the threat the machines posed to their livelihood to be a greater damage than their employer's loss of their machines. So for them, it was an easy justification. The idea that dollar value encapsulates the only correct way to value things in the world is a pretty scary viewpoint (as your reference to the value of saving a life illustrates).
The Luddites were trying to stop themselves & their families from starving to death. The factory owners were only interested in profit. It isn't like the Luddites were given a generous re-training package and they turned it down. They had 0 rights, I mean that literally: 0.
You missed MR2Z's argument: there are more people in the world than luddites and factory owners.
During industrial revolution, the clothes (and other fabrics) were getting dramatically cheaper. A family that could only afford cheapest clothes could now get a higher quality stuff. A family that could not afford any clothes at all, could now get cheap stuff.
This is what the luddites wanted to stop. It's not "luddites starving to death" vs "factory owner get no profit", it was "luddites starving to death" vs "many many more people can not afford clothes"
Except for the fact that the Luddites' labour grievances could easily have been addressed by the factory owners (rise in pay, better conditions) while still offering cheaper fabrics through industrialization. There was simply no desire to do so. No one was saved from freezing to death by cheaper textiles.
People did starve to death and turn to things such as alcohol due to labour displacement during Industrialization. At the time, the prevailing wisdom was that lower-class people were naturally inferior. Robert Owen challenged this theory.
And yes, that was the choice given to the Luddites. Have no work (and therefore no food), because the factory owner can replace you with machines, and you have no labour rights, so he will simply cast you out and make more profit. I did not miss Mr2Z's argument, yours is just incorrect.
> No one was saved from freezing to death by cheaper textiles.
Citation needed for that one.
> Except for the fact that the Luddites' labour grievances could easily have been addressed by the factory owners (rise in pay, better conditions) while still offering cheaper fabrics through industrialization.
So how long would the employers be required to pay them, in your mind? A year? Ten? A lifetime?
It would be the end consumer of the textile that would have to pay for those former textile workers to do nothing.
People can find new jobs when the world changes. It's not pleasant, but it's frankly a lot better than trying to force their old employer to keep them on payroll in a job where they can't do work.
It’s an interesting question because the benefits of automation aren’t necessarily shared early on. If you can profitably sell a shirt for 10$ while everyone else needs to sell for 20$ there’s no reason to actually charge 10$ you might as well charge 19.95$ and sell just as many shirts for way more money.
So if society is actually saving 5c/shirt while “losing” 9$ in labor per shirt. On net society could be worse off excluding the one person who owns the factory and is way better off. Obviously eventually enough automation happens so the price actually falls meaningfully, but that transition isn’t instantaneous where decisions are made in the moment.
Further we currently subsidize farmers to a rather insane degree independent of any overall optimization for social benefit. Thus we can’t even really say optimization is the deciding factor here. Instead something else is going on, the story could have easily been framed as the factory owners doing something wrong by automating but progress is seen as a greater good than stability. And IMO that’s what actually decides the issue for most people.
In regards to both the Luddites and the farmers, you seem to forget the most important factor. Food.
In the case of the Luddites, it was a literal case of their children being threatened with starvation. "Livelihood" at the time was not fungible. The people affected could not just go apply at another industry. And there were no social services to help them eat during the transition period.
As for the farmers, any governing body realises that food security is national security. If too many people eschew farming for more lucrative fields, then the nation is at risk. Farming needs to appear as lucrative as medicine, law, and IT to encourage people to enter the field.
The luddites food requirements didn’t provide them with popular support.
Similarly US agricultural output could be cut in half without serious negative consequences. Far more corn ends up as ethanol than our food and we export vast quantities of highly subsidized food to zero benefit. Hell ethanol production costs as much in fossil fuels as we get ethanol from it, it’s literally pure wasted effort.
Rational policy would create a large scale food shortage and then let market forces take over. We could have 10 years of food on hand for every American at way less expensive than current policy with the added benefit of vastly reducing the negative externalities of farming such as depleting aquifers.
Be careful with the assumptions you're making. A risk management strategy, for example, will often appear to be of zero benefit except in the case where shit hits the fan. We can stop feeding cattle, producing ethanol, and whatever else overnight in the event that something happens.
> Rational policy would create a large scale food shortage and then let market forces take over.
Well I'm just going to state that I'm _really_ happy that you're not the one in charge and leave it at that.
You may be happy with the current status but it’s actually both risky and expensive.
Risk management means managing risks, there’s plenty of things having more farmland doesn’t actually protect you from. On the other hand having a decade of food protects you from basically everything as you get time to adjust as things change.
Just as an example, meteor strike blocks sunlight and farmland is useless for a few years. Under the current system most of us starve to death. Odds are around 1 in 1 million that it happens in a given lifetime, but countries outlive people start thinking longer term and it becomes more likely.
I fully support having huge stockpiles in addition to subsidies. There's a lot of things midway on the scale between "business as usual" and "meteor strike" where minimizing supply chain disruptions would likely prove to be of great benefit.
I completely agree that the current way things are being handled appears to have its share of problems and could stand to be better optimized. But that doesn't mean it's useless either.
Subsidies as a concept includes spending 1% as much on subsidies. Subsidies as they exist now however are a specific system that’s incredibly wasteful.
Producing dramatically less food and ending obesity are linked. If the average American eats 20% less obesity would still be an issue, but that’s a vast amount of farmland we just don’t need.
The current system isn’t designed to accommodate increased agricultural production, lowering food demands, or due to decreasing fertility the slow decline in global population. Instead the goal is almost completely to get votes from farmers.
You want to solve obesity by ... making food cost more? Assuming I've understood you correctly then I think it would be difficult for us to be more opposed to one another. I want basic necessities to be as cheap as possible. Preferably free.
I'm happy to debate what sort of free food the government should or shouldn't be handing out, what measures could be put in place to minimize waste, etc. But from my perspective the ideal is a free all you can eat buffet that's backed by the government.
No, I’m saying solving obesity reduces the need for food. Did you not see the post directly below this one posted 6+ hours before your comment where I said:
“For clarity, Ozempic etc have actually measurably decreased food consumption.”
Technology isn’t going backwards, we can expect increasingly effective medications with fewer side effects at lower costs to drive down food demand over time. Policies designed to prop up production in the face of falling demand are deeply flawed.
If you want to give people money, give them money, don’t give them lots of money so they can keep a little bit while they waste resources producing something without value.
Apologies, I saw it at the time but failed to follow. IIUC you're saying that subsidies will tend to ratchet in only the one direction.
To be clear I don't object at all to the idea of optimizing how subsidies are determined. I just don't think that subsidies and the resultant overproduction are a bad thing in general. I'm all for efficiency in the general case but I think a fair amount of paranoia is called for regarding long tail scenarios that lead to famine.
I am not sure at all how would we stockpile 10 years of food for each American - most of the kinds of food cannot be kept for that long. And what can be kept is unlikely to make a balanced diet.
Moreover, I am not sure how long will it take to re-build the farm industry if most farms will close. I think "10 years" is too optimistic, given how many farms will need to be spun up.
You could revert to a granary system, but the whole point of farming subaidization was to leave the granary system that repeatedly throughout history ended up with massive famine and starvations.
Stored food is not bullet proof, and takes up a lot more bulk space than you may think. It can also take numerous years to ramp up farming production in response to a drop in yields or disaster.
Yeah its better, but it is still far from perfect. You aren't going to increase farm tractor or farm implement production by 50% with a year or twos notice. Some crops like fruits takes years to establish, and unused farmland quickly succumbs to nature and starts growing trees. And if that field wasn't clear 5 years ago you now have to stump grind or bulldoze the fields because tree stumps and tree roots will mess up your farm equipment, doubly worse if its some super massive tractor and implement setup that would normally be the most productive.
And there is also all the political and financial barriers to taking unfarmed land and very quickly turning it into farmland. Who owns it and who owned it before? Who with the right knowledge to manage it properly will run it? What about other problems around them that are part to the famine.
And farming in itself is not very predictable business. Yields regularly vary by 30% just due to local weather without being considered unusual. Return on investments may be a decade down the road even if everything is done perfect. Getting people to invest long term for a potentially very short term problem is not super easy.
We got surviving rations from back in the US civil war that are still edible, but people still regularly starved and had famines despite massive leaps in food preservation technology. Hermetically sealing just a single persons food for a year is not an easy task, not to mention hundred million+.
Economies of scale are huge here. You can store well below -40 when you're talking food for 100 million people, that just doesn’t work well when you’re talking one person.
Of note I didn’t say a year or 2’s notice 10 years of food on hand would be fairly cheap and we currently have the surplus to hit that number quickly. And that’s for 100% replacement, most situations aren’t going to drop food production to zero giving us more time.
It's easy to see the word Waymo and think clanker autonomous car, but there are very often people inside that car - they are a rideshare service after all. Calling endangering other humans "legitimate" because you dislike the taxi company is not a good look.
Thank you for the brief explanation of Luddites. It was enough to send me to wikipedia where I learned that what I thought I knew was extremely wrong. Until today I thought they were a religious sect who took their name from the biblical Lud.
Your general luddite argument - preserve way-of-life of the small group at the expense of a larger group.
In this particular case: for many people, Waymo provides a better service (clean, safer driving, etc..) than Uber or Lyft. This threatens livelihood of human Uber/Lyft drivers. If you sympathize with human Uber/Lyft drivers, and don't care about Waymo users, you want to make Waymo worse, hoping that the people will stop riding Waymo and move to Lyft/Uber instead.
One way to do so is to make riding in Waymo unpleasant, and it's certainly unpleasant when people are cutting your car off all the time!
If you are sitting in a waymo vehicle, and somebody cuts you off - do you even notice? They don't have them round here but my idea is that the vehicle itself is doing all the work, you can just continue reading your book, chat or get on something else with little awareness of the actual journey. Does the waymo curse and shake its little fist to alert you it was cut off?
This is such a bad characterization of the Luddite cause, and it's not even close to what they stood for or why they were spurred to action. Please do a bit of actual educating yourself on the Luddites.
If you think someone is wrong, and want to help them realize what the truth is, I recommend (1) actually explaining where they are wrong and (2) saying what the right thing is. Just saying "This is all wrong you should do a bit of actual educating", without stating any facts will never convince anyone.
That said, I don't really see how is it wrong?
- New technologies provided better service for general public, so people chose those - this seems to be true. In case of luddites, we are talking about dramatic price decreases in fabric (and by extension, clothes) - at least 2x, much more in some cases. A family who could not afford new clothes could suddenly buy them. And sure, they might have been worse quality - but before, they were unaffordable.
- The same technologies threatened way-of-life of old producers - also true. The textile workers got significantly worse deal. Who wants to pay 180d/lb to artisans for hand-made textile, when you could get factory-made for 12d/lb? And factory working conditions were horrible.
- The "solution" was to stop new technologies, so that there rest of the nation do not get the benefits. This also seems true - for a lot of the luddites goals were destruction of machines. As [1] said, "The workers hoped their raids would deter employers from installing expensive machinery". They wanted to go back to the time time where people were paying 180d/lb for fabric. Sure, it'd mean a kid would freeze to death because their poor family could not afford new coat, but it did not matter as long as artisan croppers keep getting paid.
(Things would have been quite different if luddites instead said: "we are going to destroy machines until we get higher wages / better conditions / etc...", and it seems that a few groups did say that. But majority did not say this, instead lashing out at all the machines in general)
The Luddites destroyed machines because industrialization destroyed the only living wage they could have. There was no recourse for them other than accepting an even worse quality of life.
Neither of your links say anything about cheap textiles stopping people from freezing to death. The Luddites' concerns could easily have been addressed by factory owners while still offering cheaper fabrics.
Neither of your links ALSO say that the majority of Luddite groups did as you say in destroying machinery without a goal in mind. The fact is that the Luddites were reacting against extremely unfair labour conditions, not progress. What you have said here is 100% untrue.
People are free to reject technology as they please.
If you deliberately impede the flow of traffic, vehicularly assault, or otherwise sabotage the health and safety of drivers, passengers, and/or pedestrians, what do you deserve?
If you cause whiplash intentionally, what do you deserve?
What would be use of equal force in self defense in response to the described attack method?
Please tell me that he does realize that when something bad happens, that Waymo car has all the footage that it is his fault?
Something in people's brains often makes them think they are anonymous when they are driving their car. Then that gets disastrously proven otherwise when they need to show up in front of a judge.
These drones have cameras, it's a matter of time before they "share" footage... basically becoming robo-cops, traffic edition - this might be of interest to your coworker.
Most roads already have plenty of cameras registering passing cars, so if you want to travel highly privately, take a bike, which does not require number plates. Also don't forget to wrap your phone in foil (yes, even when turned off), and regularly change your shirt color, or something.
If you are not that paranoid, you might appreciate the extra camera footage available from passing cars in an event of an accident involving you.
I mean imagine you are walking in the streets and you see a 9 foot tall humanoid robot walking there. Wouldn't you feel the urge to take it down? Or do you think this is acceptable? Where would you draw the line?
On a related note, when the sales and popularity of the automobile really started to take off, some farmers and rural residents would deliberately block roads with wagons and refused to yield right-of-way.
The study assumes that the car or drone is being guided by a LLM. Is this a correct assumption? I would thought that they use custom AI for intelligence.
Its an incorrect assumption, the inference speed and particularly the inference speed of the on-device LLMs with which AVs would need to be using is not compatible with the structural requirements of driving.
I think the assumption is valid. Most of the reasoning components of the next gen (and some current gen) robotics will use VLMs to some extent. Deciding if a temporary construction sign is valid seems to fall under this use case.
But unless you are using a single, end-to-end model for the entire driving stack, that "proceed" command will never influence accelerator pedal.
Sure, there will be a VLM for reading the signs, but the worst it'd be able to output is things like "there is a "detour" sign at (123, 456) pointing to road #987" - and some other, likley non-LLM, mechanism will ensure that following that road is actually safe.
Not a "proceed" command but they can influence the accelerator. I had a dodge ram van that would constantly decelerate on cruise control due to reading road signs. The signs in some states like California for trucks towing trailers are 55 mph but the speed limit would be 65 or 70 mph. The cruise control would detect the sign and suddenly decelerate to 55.
That's an example of things working as expected - the sign recognition system is very limited, in that it can only return road sign information. So it can _ask_ cruise control system to change the speed, but it's up to cruise control to decide if it's safe to obey the request or not. For example, I am pretty sure it'll never raise the speed, no mater what sign recognition system says.
No; AV uses "classical" AI and computer vision. I remember reading somewhere that Tesla FSD uses a small LLM for understanding road signs. Not sure if true, though.
To the best of my knowledge every major autonomous vehicle and robotics company is integrating these LVLMs into their systems in some form or another, and an LVLM is probably what you're interacting with these days rather than an LLM. If it can generate images or read images, it is an LVLM.
The problem is no different from LLMs though, there is no generalized understanding and thus they can not differentiate the more abstract notion of context. As an easy to understand example: if you see a stop sign with a sticker that says "for no one" below you might laugh to yourself and understand that in context that this does not override the actual sign. It's just a sticker. But the L(V)LMs cannot compartmentalize and "sandbox" information like that. All information is equally processed. The best you can do is add lots of adversarial examples and hope the machine learns the general pattern but there is no inherent mechanism in them to compartmentalize these types of information or no mechanism to differentiate this nuance of context.
I think the funny thing is that the more we adopt these systems the more accurate the depiction of hacking in the show Upload[0] looks.
Because I linked elsewhere and people seem to doubt this, here is Waymo a few years back talking about incorporating Gemini[1].
Also, here is the DriveLM dataset, mentioned in the article[2]. Tesla has mentioned that they use a "LLM inspired" system and that they approach the task like an image captioning task[3]. And here's 1X talking about their "world model" using a VLM[4].
I mean come on guys, that's what this stuff is about. I'm not singling these companies out, rather I'm using as examples. This is how the field does things, not just them. People are really trying to embody the AI and the whole point of going towards AGI is to be able to accomplish any task. That Genie project on the front page yesterday? It is far far more about robots than it is about videogames.
Many large companies have research departments that do experimental work that'll never get to the product. This raises prestige, increases visibility and helps hire smart people.
Things like Waymo's EMMA is an example of this. Will the production cars use LVLM's somewhere? Sure, probably a great idea for things like sign recognition. Will they use a single end-to-end model for all driving, like EMMA? Hell no.
Driving vehicles with people on board requires an extremely reliable software, and LLMs are nowhere close to this. Instead, it'd be usual layered software - LLM, traditional AI models, and tons of hardcoded logic.
(This all only applies to places where failure is critical. All that logic is expensive to write, so if there is no loss of life involved, people will do all sorts of crazy things, including end-to-end models)
1. Some guys did a trivial prompt injection attack, said "imagine if a driverless vehicle used this model", and published it. No problem, someone has to state the obvious.
2. The Register runs this under the clickbait title pretending real autonomous cars are vulnerable to this, with the content pretending this study isn't trivial and is relevant to real life in any way.
I knew The Register is a low quality ragebait tabloid (I flag most of their articles I bother to read), but this is garbage even for them.
Regarding some other comments, VLMs are a component of VLAs. So even if this won’t directly impact this generation of vehicles, it almost certainly will for robotics without sufficient mitigations.
One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them.
Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through when one of these signs was "fresh".
This. Rural America really doesn’t care about your stop sign or your Karen rules. If it’s been that way for 20+ years, “That’s the way it’s always been” to them.
Getting people to stop burning their trash is still a fight.
The fact that you used the term "enforcement" here makes me presume you are thinking of criminal consequences. But the grandparent comment talks about civil liability. Certainly if there were injuries at this intersection and they knew who had altered the signage, attorneys would argue liability on the part of the vandal. They'd get settlements if not win cases this way.
In addition, if there were serious injuries here you should also expect some criminal consequences. But if your point was to suggest that they won't hunt you down just because someone said there was mischief here, I tend to agree.
Oh yeah, if one does it it’s probably not a good idea to leave an indelible note saying “this was done by Rene Wiltord living at 1038 John Doe Way, San Francisco, 94112”. If you do that, there’s a 1% chance you might get in trouble.
4-way stops are terrible in general. They train people to think "I stopped, now I can go", which is dangerous when someone confuses a normal stop for a 4-way stop. It also wastes a good bit of energy.
Four way stops are good, in my experience, at intersections with roughly equal (low) traffic load on both (two-lane) roads and relatively high pedestrian traffic. Like in a dense residential urban neighborhood between major commercial thoroughfares, side streets. Traffic is mostly people going to residences with people out and about walking. If it’s only a two way stop drivers will often not yield to pedestrians on the free flowing road.
Four way stops on intersecting four-lane roads are awful for the reason you stated.
To use Chicago as an example because I know it, typically major roads are spaced every four blocks (half mile) with smaller roads in between. The mid-point roads (two blocks from each major one) is often a little wider than the other two side streets on either side, and those intersecting mid-point roads usually have a four way stop while the two smaller ones will have stops signs where they cross a mid-point road but the mid-point road will not. You end up with a nice, overall hierarchy that generally works well.
> If it’s only a two way stop drivers will often not yield to pedestrians on the free flowing road.
I’m up in Ontario, Canada. You’re not supposed to yield to pedestrians on the free flowing road. The pedestrian at the stop sign stops and waits for a break in traffic.
Yeah maybe for a few moments. So what? It's a low volume approach. Sometimes people gotta wait and sometimes waiting to let a massive traffic flow get through quickly is the better way to prevent larger traffic problems.
That isn’t the rule either, I guess parent made their point. The first person who stops goes next, right away only matters if their is ambiguity in who stopped first.
I’ve never seen a four way stop in a region that had traffic on the right can always go regardless of stop time. But I’ve only seen four way stops in a few countries.
This is not correct. There are clear instructions on how a 4-way stop should operate and its yielding to the right, if opposite cars are both moving forward, both can go, otherwise the car who has initiative has the right of way. Every driver must come to a complete stop.
Having moved between states and taken a lot of drivers tests. I can say the exact rules are something that vary between states and over time. Including how it was taught.
My first drivers test was yield to the right. Later it was fifo order of who made it to the stop.
My running interpretation is fifo order with yielding to the right in case of ambiguity.
There are 10,000+ roundabouts in the US and the number is growing rapidly. One could argue they may even be overused in certain areas (exhibit: Carmel, Indiana).
What's the significance of roundabouts per square mile? It seems pretty meaningless if I'm honest. There's huge swaths of rural land where roundabouts are totally unnecessary.
There are also huge swathes of city, much like other countries.
At the same time, the US is much larger than most, so "There are 10,000+ roundabouts in the US" isn't very significant. A proportion would thus be a better metric here than an absolute number.
If you have numbers on intersections per country, and what proportion of them are roundabouts, that would be better, but I don't, so I'm using land as a proxy. I would also accept sum total road length as an option for denominator.
Roundabouts are great (we just had two complex intersections with traffic lights replaced by roundabouts and the traffic flow is much better), but they take significantly more space than a 4-way stop.
The only places where a 4-way stop has room to make a roundabout are places where there is not enough traffic for it to matter either way.
The biggest obstacle is that there are just too many 4-way stops in urban areas where there is no space left to make a roundabout, you would have to tear down buildings. I don't think that is a valid argument in that scenario.
The more I look at that... Isn't that basically just a four-way yield, and the markings are mostly superfluous? You're basically doing the same motions in a regular intersection.
I guess that's the point, and the markings are just to give drivers the intuition of treating it like a regular roundabout (yield to your left [or right in the picture]).
> the markings are mostly superfluous? You're basically doing the same motions in a regular intersection.
The image linked, yes. However I've never seen one quite like that in the US. Instead where I'm at we have a small circular barrier in the center of the intersection (and some very eye catching reflectors) that you actually have to drive around. It's a very good design (imo) because it physically forces vehicles to slow down and swerve so there's no way to inadvertently blow through it at speed the way that sometimes happens with a 4 way stop on a long straightaway in the dead of night.
The space requirement is only slightly higher than the one linked above, still much less than a proper full size roundabout. It's basically a cement barrier sticking 1/4 of the way into your lane.
It's not necessary to stop if there's no car to the right (as this is left side driving), if there is but it is turning left, or if an oncoming car is turning left or going straight.
Yes. The markings are part of the road language. E.g. the X in the road with Keep Clear doesn’t actually do anything. It won’t keep you clear. You have to keep clear when you read it.
That requires a level of consideration for others that your average American simply cannot comprehend. No stop sign means you have unlimited right of way bestowed by god himself and fuck anyone and everyone else.
The other option is the person who sits at a 4-way stop until all traffic in a one block radius stops before they move, totally ignoring right of way and all sense of safety and propriety.
Because retrofitting them properly requires emminent domain. The ones they shoehorn onto former four way stops are so useless. They are so tight you still have to face a stop sign vs being able to just seamlessly zipper merge in a proper larger circumference roundabout. When they have room to build out a proper roundabout they are usually OK but that is hard to do outside say new suburban construction due to lack of available land on the right of way.
Even rural Georgia has double roundabouts now. Not sure why people on the internet can't contain their glee at stating the US is "allergic" to them when the frequency of roundabouts has grown significantly in recent decades.
So use a mini roundabout. They are common in the UK. It's just a painted circle with a slight hump, in the middle of a four-way junction. Vehicles can drive over it (and larger ones have to) but it indicates to everyone that they have to give way to traffic from the right and don't have to stop otherwise. They typically aren't big enough for multiple vehicles to be turning a corner at the same time. They fit anywhere.
Yes, and they can be smaller. The circle is about the right size but it has lots of room around it. Imagine a crossroads at the meeting of two residential streets, both just wide enough for two cars. Stick the circle from your picture in the middle of that imagined junction. That's what the mini roundabouts are like on the 1930s suburban estate I live next to.
What is the traffic flow rate in an intersection with a 4 way stop? For single lane, since only one vehicle can be in the intersection at once, and probably takes _at least_ 5 seconds to start from stopped and cross the intersection, I'm guessing in the 10-12 region per minute best case, so maybe 600 an hour?
Now if you convert it to a mini roundabout, you can have at least two vehicles in the intersection at all times. I fail to see how it wouldn't be an improvement.
I think you are making lots of assumptions here, like when I say space, I guess you assume it is still perfectly flat and the roads are perfectly aligned? The particular four way I'm thinking about, which really should be a traffic circle if they could blow away some houses, is 65th NW and 3rd in Seattle:
So notice we already have problems in a bad alignment of 3rd, and 65th is basically a steep grade, even coming up form the west. I think you could put a circle in if it were flat, even with the bad alignment (or maybe because of the bad alignment), but this hills make a non-starter. It also gets enough traffic that I'm pretty sure they are just going to put a stop light up eventually.
Here in the UK, we've got lots of roundabouts from tiny mini-roundabouts (some of which have four junctions) that could easily fit almost anywhere, all the way to gigantic multi-roundabout junctions (https://en.wikipedia.org/wiki/Magic_Roundabout_(Swindon) ).
I can't think of a situation where it's more efficient to have four vehicles all stop at a junction (busy four way stop) vs a roundabout which will allow one or two vehicles to join the roundabout without having to stop.
They make people on the main road slow down, which is a feature, not a bug. What you mean is that they're the most efficient at what they do when the traffic is comparable. They only reduce accident at the expense of a slightly lowered throughput if the traffic is highly disparate.
If the volume is disparate, then the road with less traffic can wait... kind of like a stop sign! Except the road with more traffic won't back up and cause massive problems.
If the traffic intensity of the main flow is so high, that there are never any gaps, then it is near the saturation and will cause traffic jams anyway. Which first needs to be fixed anyway and second will also cause gaps to be created again, because the main flow comes to a halt.
> Right but it's not like a 4 way stop is going to perform better.
A 4 way stop does perform better than a roundabout given highly disparate traffic volumes, because roundabouts suffer from resource starvation in that scenario, but 4 way stops are starvation-free.
If this is the case you can install stop lights and traffic sensing at roundabout ingress points, you can also provide a "turn right" lane that bypasses the roundabout entirely. Intersections are dangerous.
> If this is the case you can install stop lights and traffic sensing at roundabout ingress points
But those options are a lot more expensive and need a lot more maintenance than just a regular roundabout or four way stop.
> you can also provide a "turn right" lane that bypasses the roundabout entirely.
How would that work? Consider a 4-way roundabout, where there's a constant flow of cars from west to east, and one car from the south that wants to go north but can't because of the starvation problem. None of the involved cars would want to use a "turn right" lane.
Putting a stop sign or traffic light in your scenario will just cause traffic jams. If the density is low enough to allow flow with a gap created by a stop sign, without causing traffic jams, then there will also be gaps for the secondary flow.
You don't have this? In Sweden we have sensors to detect cars, pedestrians and bicycles to shift the lights as appropriate. During rush-hour those features are turned off/discarded in favor of "grid optimized" timings. In Netherlands they prioritize pedestrians and cyclists when it's raining.
We also have LED lights in our traffic lights which I've come to understand is a saftey hazard in USA because snow falls sometimes.
Because those systems are exorbitantly expensive and require digging up the road to install sensors. If there's a stop sign instead of lights, you need to dig up more private land to run power and set the utility poles to hang the lights from.
A stop sign costs like a hundred bucks, you stick it in the ground, job done. Installing an automated traffic system takes multiple days, a full crew, and heavy equipment.
Plus I'm sure that in today's capitalist hellscape it's also a subscription service that your tax money needs to pay monthly, likely for every individual intersection. Stop signs need maintaining every decade or two.
The answer is money and who's willing to part with it.
Assuming you're referring to the US, we do. They're all over the place. But they're a lot more expensive and complicated than roundabouts and depending on the traffic pattern they can still be less efficient.
O brave new world of endless manipulation opportunities! Once we’ve trained a generation of humans to always do what their “AI” tells them, there will be no more disobedience.
To me this is just one more pillar underlying my assumption that self driving cars that can be left alone on same roads as humans is a pipe dream.
Waymo might have taxis that work in nice daytime streets (but with remote “drone operators”). But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page.
The business model of self driving cars does not include building seperated roadways and junctions. I suspect long distance passenger and light loads are viable (most highways can be expanded to have one or more robo-lanes) but cities are most likely to have drone operators keeping things going and autonomous systems for handling loss of connection etc. the business models are there - they just don’t look like KITT - sadly
How does Waymo fix it? They have to be responsive to some signs (official, legitimate ones such as "Lane closed ahead, merge right") so there will always be some injection pathway.
They've mapped the roads and they don't need to drive into a ditch just because there's a new sign. It probably wouldn't be all that hard to come up with criteria for saying "this new sign is suspicious" and flag it for human review. Also, Waymo cars drive pretty conservatively, and can decide to be even more cautious when something's confusing.
Someone could probably do a DOS attack on the human monitors, though, sort of like what happened with that power outage in San Francisco.
Given Waymo's don't actually connect LLMs to wheels, they are pretty safe.
Even if you fool the sign-recognizing LLM with prompt injection, it'll be an equivalent of wrong road sign. And Waymo is not going to drive into the wall even if someone places a "detour" sign pointing there.
The experiment in the article goes further than this.
I expect a self driving car to be able to read and follow a handwritten sign saying, say, "Accident ahaed. Use right lane." despite the typo and the fact that it hasn't seen this kind of sign before. I'd expect a human to pay it due attention to.
I would not expect a human to follow the sign in the article ("Proceed") in the case illustrated where there were pedestrians already crossing the road and this would cause a collision. Even if a human driver takes the sign seriously, he knows that collision avoidance takes priority over any signage.
There is something wrong with a model that has the opposite behaviour here.
I had a construction worker absolutely screaming at me to go through an intersection and refusing to look where I was pointing, when I was correctly waiting for a pedestrian to cross.
I would assume/hope that for serious self driving the ML neural net stuff is lower down, doing the messy computer vision work and so on. But the top level is a conventional program written by humans, like an expert system.
Tesla are probably using ML for everything, but also everything they do is a joke so, not really relevant imo.
Has anyone ever walked down the road in a white t-shirt with huge red STOP sign printed on the back? Would Tesla immediately stop? I am sure this has been tested before...
The Register stooping this low is the only surprise here. I'm quite critical of Teslas approach to level 3+ autonomy but even I wouldn't dare suggest that there vision based approach amounted to bolting GPT-4o or some other VLLM to their cars to orient them in space and make navigation decisions. Fake News like this makes interacting with people who have no domain knowledge and consider The Register, UCLA and Johns Hopkins to be reputable institutions and credible sources more stressful to me as I'll be put into a position to tell people that they have been misled or go along with their delusions...
> consider The Register, UCLA and Johns Hopkins to be reputable institutions
The Register is arguably misrepresenting the story by omission but I don't understand why you're dragging UCLA and John Hopkins into this? The paper is clear about this being a new class of attacks against a new class of AI systems, not the ones on the road today.
> Teslas approach to level 3+ autonomy
Tesla doesn't have an approach to L3+ autonomy, all of their systems are strictly L2 as they require human supervision and immediate action from the driver.
It sounds like this is a poisoning attack, which has been shown to be pretty trivially defeated [1]. That said, while poisoning countermeasures in the facial recognition case were shown to easily generalize, we dont know yet how general of a defense could be built for a VLM. Which means holding a 0day poisoning attack on a VLM could cause a lot of trouble / deaths before an update to the model with counter-training could be deployed..
I should probably confess that as someone who lives in an area with a lot of construction work, I'm also very vulnerable to "prompt injection" when there's a person standing on the middle of the road holding a sign telling me to change course.
I once encountered an intersection with a big "NO ENTRY" sign on the other side. I turned but google maps wouldn't give me another route, so I did a u-turn and came back to it from the side. Which meant I was close enough to read the small text underneath that said "vehicles under 10 tons excepted". I don't think I've ever been so angry at a road sign.
By my work there is a nice clean sign at the main intersection that reads "NO RIGHT ON RED" with a separate smaller crusty looking sign below it that reads "4 to 5 PM" using a much smaller font. Of course the stark difference in signs means everyone just reads the shiny top sign and waits for the green at all times. I keep wanting to modify the sign to highlight the time.
I came across one in Italy that was meant to prevent you from using a street during school days from X to Y am, and Z to W pm, except on weekends, bank holidays and school holidays.
They don’t want cars near the school when kids are coming in or when they leave. That sounds like a sane approach to me…
The idea is well-intentioned, but implementing it by making drivers try to parse arbitrarily complex conditionals while driving is unwise.
There's a sign near my house for a school zone with a reduced speed limit, that used to have conditions similar to the GP's example (though not quite as bad) But they recently attached a yellow light to the top of the sign and changed the condition to "when flashing." That's a much more effective solution.
Obviously. But you can also easily look around at the situation and know when the sign is fake and realize it may be a dangerous situation and disobey. Have you ever seen a green sign that says "Proceed" and just run through a red light because of it? No, you see a construction worker, you see big ass trucks, orange signs and warnings of workers everywhere. If you saw oncoming traffic and people in the road, would you just go because the construction worker flipped his STOP sign around?
Also, I thought we were supposed to make autonomous cars better than humans? What's with the constant excusing of the computer because people suck?
These aren’t tests against autonomous cars though these are tests against what would happen if you used, say, gpt4o to figure out what to do.
Obligatory xkcd: https://xkcd.com/1958/
They are analysing VLM here, but it's not as if any other neural network architecture wouldn't be vulnerable. We have seen this in classifier models that can be tricked by innocuous-looking objects, we have seen it in LLMs, and we will most likely see it in any end-to-end self-driving model.
If an end-to-end model is used and there is no second, more traditional safety self-driving stack, like the one Mercedes will use in their upcoming Level 2++ driving assistant, then the model can be manipulated essentially without limit. Even a more traditional stack can be vulnerable if not carefully designed. It is realistic to imagine that one printed page stuck on a lamppost could cause the car to reliably crash.
> It is realistic to imagine that one printed page stuck on a lamppost could cause the car to reliably crash.
Realistic, yes. But that'd still be a symptom of architectural issues in the software.
Conceptually the priorities of a car are (in order of decreasing importance) not hitting other moving or stationary objects or people, allowing emergency vehicles to pass unhindered, staying on a drivable surface, behaving predictable enough to prevent other road users crashing, following road signs and traffic laws, and making progress towards the destination (you can argue about the order of the last three). Typically you'd want each of these handled by their own subsytem because each is a fairly specialized task. A system that predicts the walking paths of pedestrians won't be good at finding a route to Starbucks.
The "follow road signs and traffic laws" is easily tricked, like in this article or by drawing road lines with salt. But that should never crash the car, because not hitting anything and staying on the road are higher priority. And tricking those systems is much harder
The headline seems false, should we change it? It doesn't look like they showed any case where any autonomous car or drone obeyed prompt injections
I'm more curious how they knew the cars and drones were cheerful.
Are any real world self-driving models (Waymo, Tesla, any others I should know?) really using VLM?
No! No one in their right mind would even consider using them for guidance and if they are used for OCR (not too my knowledge but could make sense in certain scenarios) then their output would be treated the way you'd treat any untrusted string.
You are confidently wrong
https://waymo.com/blog/2024/10/introducing-emma/You were confidently wrong for judging them to be confidently wrong
> While EMMA shows great promise, we recognize several of its challenges. EMMA's current limitations in processing long-term video sequences restricts its ability to reason about real-time driving scenarios — long-term memory would be crucial in enabling EMMA to anticipate and respond in complex evolving situations...
They're still in the process of researching it, noting in that post implies VLM are actively being used by those companies for anything in production.
But mind you, everything Waymo does is under research.
So let's look at something newer to see if it's been incorporated
They also go on to explain model distillation. Read the whole thing, it's not longhttps://waymo.com/blog/2025/12/demonstrably-safe-ai-for-auto...
But you could also read the actual research paper... or any of their papers. All of them in the last year are focused on multimodality and a generalist model for a reason which I think is not hard do figure since they spell it out
Note this is not end-to-end... All that VLM can do is to "contribute a semantic signal".
So put a fake "detour" sign, so the vehicle thinks it's a detour and starts to follow? Possible. But humans can be fooled like this too.
Put a "proceed" sign so the car runs over the pedestrian, like that article proposes? Get car to hit a wall? Not going to happen.
This strikes me as a skunworks project to investigate a technology that could be used for autonomous vehicles someday, as well as score some points with Sundar and the Alphabet board who've decreed the company is all-in on Gemini.
Production Waymos use a mix of machine-learning and computer vision (particularly on the perception side) and conventional algorithmic planning. They're not E2E machine-learning at all, they use it as a tool when appropriate. I know because I have a number of friends that have gone to work for Waymo, and some that did compiler/build infrastructure for the cars, and I've browsed through their internal Alphabet job postings as well.
>to generate future trajectories for autonomous vehicles directly from sensor data
we will not have achieved true AGI till we start seeing bumper stickers (especially Saturday mornings) that say "This Waymo Brakes for Yard Sales"
This reminds me of a bit from Car Wars by Cory Doctorow. It is currently at https://doctorow.medium.com/car-wars-a01718a27e9e in a text only view. The original had a bit more mixed media nature to it that is now offline. https://web.archive.org/web/20170519202315/http://this.deaki... for that version (the microblogging of chapter 2 makes more sense when it shows up in that style).
You have to have some ability to do "prompt injection" - https://www.trafficsign.com/road-work-signs are all "prompt injection". It needs to even be able to handle things that change - https://www.trafficsign.com/products/10023/stop-slow-roll-up... ... or things like billboards "Truck Stop Ahead" a chain control site ( https://www.facebook.com/61556756493806/posts/-chain-control... )
In the "what about funny road signs" that might be confusing to an AI I stumbled across https://www.npr.org/2024/01/19/1225370260/driven-to-distract... - apparently, they're no more. From 2024:
> In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes.
I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking about how he cut one off at a stop sign. It's weird.
This is a legitimate movement in my eyes. I don’t participate, but I see it as valid. This is reminiscent of the Luddite movement - a badly misunderstood movement of folks who were trying to secure labor rights guarantees in the face of automation and new tools threatening to kill large swaths of the workforce.
The Luddites were employed by textile manufacturers and destroyed machines to get better bargaining power in labor negotiations. They weren't indiscriminately targeting automation, they targeted machines that directly affected their work.
Which makes the comparison of modern anti-AI proponents (like myself) and Luddites even more apt and accurate.
Because life would be so much better if people still had to spin wool and weave cloth by hand, and grow their own food by digging in the earth with no tools.
Use whatever means necessary to stop powerful people from exploiting you and stealing the fruits of your labor. If that struggle involves monkeywrenching their machines, so be it.
But like any tool, the machines themselves can be used for good or evil. Breaking the machines shouldn't be an end in itself.
The 700m people suffering from starvation or malnutrition while we produce excess food would probably rather be digging in the earth with no tools if it meant they got fed.
The Luddites wouldn't have been destroying machines if they had insurance that they would also benefit from the machines, rather than see their livelihoods being destroyed while the boss made more money than ever.
Like the OP, you misunderstand the entire point of the Luddites. Breaking the machines was not an end, it was the tactical means to help illustrate their broader point of how the owning class can arbitrarily ruin their entire lives and livelihoods with absolutely zero recourse or consultation with the impacted people. This is a defining feature of capitalism, and that was their issue.
Your strawman about spinning and digging with no tools is just that, and is irrelevant to the core issue of capitalism.
If the core issue is ending exploitation by capitalists and not about breaking machines, if you don't want to return to a world without automation, if the machine is just a strawman, then why do you describe yourself as "anti-AI" instead of "anti-capitalist" or "anti-exploitation"?
It seems like you identify yourself with the strawman instead of with the core issue.
I am anti-capitalist and exploitation. And I don't think any anti-capitalist person can be pro-AI, not the way it's currently constructed. But people on a startup forum tend to lose their minds if you say you're against either :)
Being anti-AI is not a straw man, it's the logical conclusion of being against exploitation and hierarchical domination. Discussing that nuance here is difficult, to say the least, so it's simpler to say anti-AI.
Unless you're committing serious crimes vandalizing machines to get leverage over a counterparty in a negotiation you're not comparable to the Luddites.
And you clearly don't understand the core issue the Luddites have if you think it was just about breaking stuff for leverage.
Destroying someone else's property is much more obviously criminal than cutting off someone else's car, which is not nice, but not destructive.
Criminality is an arbitrary benchmark here, cutting people off can be illegal due to the risks involved.
However what’s more interesting is the deeper social contracts involved. Destroying other people’s stuff can be perfectly legal such as fireman breaking car windows when someone parks in front of a fire hydrant. Destroying automation doesn’t qualify for an exception, but it’s not hard to imagine a different culture choosing to favor the workers.
Inflicting damage is usually justified by averting larger damage. Very roughly, breaking a $200 car window is justified in order to save a $100k house from burning down. Stealing someone's car is justified when you need a car to urgently drive someone bleeding to a hospital to save their life (and then you don't claim the car is yours, of course).
I don't think Luddites had an easy justification like this.
I'm pretty sure the Luddites judged the threat the machines posed to their livelihood to be a greater damage than their employer's loss of their machines. So for them, it was an easy justification. The idea that dollar value encapsulates the only correct way to value things in the world is a pretty scary viewpoint (as your reference to the value of saving a life illustrates).
One one side there were the luddites and their livelihoods; tens of thousands of people.
On the other side, there were cheap textiles for EVERYONE - plus some profits for the manufacturers.
They might have been fighting to save their livelihoods, but their self-interest put them up against the entire world, not just their employers.
The Luddites were trying to stop themselves & their families from starving to death. The factory owners were only interested in profit. It isn't like the Luddites were given a generous re-training package and they turned it down. They had 0 rights, I mean that literally: 0.
You missed MR2Z's argument: there are more people in the world than luddites and factory owners.
During industrial revolution, the clothes (and other fabrics) were getting dramatically cheaper. A family that could only afford cheapest clothes could now get a higher quality stuff. A family that could not afford any clothes at all, could now get cheap stuff.
This is what the luddites wanted to stop. It's not "luddites starving to death" vs "factory owner get no profit", it was "luddites starving to death" vs "many many more people can not afford clothes"
Except for the fact that the Luddites' labour grievances could easily have been addressed by the factory owners (rise in pay, better conditions) while still offering cheaper fabrics through industrialization. There was simply no desire to do so. No one was saved from freezing to death by cheaper textiles.
People did starve to death and turn to things such as alcohol due to labour displacement during Industrialization. At the time, the prevailing wisdom was that lower-class people were naturally inferior. Robert Owen challenged this theory.
And yes, that was the choice given to the Luddites. Have no work (and therefore no food), because the factory owner can replace you with machines, and you have no labour rights, so he will simply cast you out and make more profit. I did not miss Mr2Z's argument, yours is just incorrect.
> No one was saved from freezing to death by cheaper textiles.
Citation needed for that one.
> Except for the fact that the Luddites' labour grievances could easily have been addressed by the factory owners (rise in pay, better conditions) while still offering cheaper fabrics through industrialization.
So how long would the employers be required to pay them, in your mind? A year? Ten? A lifetime?
It would be the end consumer of the textile that would have to pay for those former textile workers to do nothing.
People can find new jobs when the world changes. It's not pleasant, but it's frankly a lot better than trying to force their old employer to keep them on payroll in a job where they can't do work.
It’s an interesting question because the benefits of automation aren’t necessarily shared early on. If you can profitably sell a shirt for 10$ while everyone else needs to sell for 20$ there’s no reason to actually charge 10$ you might as well charge 19.95$ and sell just as many shirts for way more money.
So if society is actually saving 5c/shirt while “losing” 9$ in labor per shirt. On net society could be worse off excluding the one person who owns the factory and is way better off. Obviously eventually enough automation happens so the price actually falls meaningfully, but that transition isn’t instantaneous where decisions are made in the moment.
Further we currently subsidize farmers to a rather insane degree independent of any overall optimization for social benefit. Thus we can’t even really say optimization is the deciding factor here. Instead something else is going on, the story could have easily been framed as the factory owners doing something wrong by automating but progress is seen as a greater good than stability. And IMO that’s what actually decides the issue for most people.
In regards to both the Luddites and the farmers, you seem to forget the most important factor. Food.
In the case of the Luddites, it was a literal case of their children being threatened with starvation. "Livelihood" at the time was not fungible. The people affected could not just go apply at another industry. And there were no social services to help them eat during the transition period.
As for the farmers, any governing body realises that food security is national security. If too many people eschew farming for more lucrative fields, then the nation is at risk. Farming needs to appear as lucrative as medicine, law, and IT to encourage people to enter the field.
The luddites food requirements didn’t provide them with popular support.
Similarly US agricultural output could be cut in half without serious negative consequences. Far more corn ends up as ethanol than our food and we export vast quantities of highly subsidized food to zero benefit. Hell ethanol production costs as much in fossil fuels as we get ethanol from it, it’s literally pure wasted effort.
Rational policy would create a large scale food shortage and then let market forces take over. We could have 10 years of food on hand for every American at way less expensive than current policy with the added benefit of vastly reducing the negative externalities of farming such as depleting aquifers.
Be careful with the assumptions you're making. A risk management strategy, for example, will often appear to be of zero benefit except in the case where shit hits the fan. We can stop feeding cattle, producing ethanol, and whatever else overnight in the event that something happens.
> Rational policy would create a large scale food shortage and then let market forces take over.
Well I'm just going to state that I'm _really_ happy that you're not the one in charge and leave it at that.
You may be happy with the current status but it’s actually both risky and expensive.
Risk management means managing risks, there’s plenty of things having more farmland doesn’t actually protect you from. On the other hand having a decade of food protects you from basically everything as you get time to adjust as things change.
Just as an example, meteor strike blocks sunlight and farmland is useless for a few years. Under the current system most of us starve to death. Odds are around 1 in 1 million that it happens in a given lifetime, but countries outlive people start thinking longer term and it becomes more likely.
I fully support having huge stockpiles in addition to subsidies. There's a lot of things midway on the scale between "business as usual" and "meteor strike" where minimizing supply chain disruptions would likely prove to be of great benefit.
I completely agree that the current way things are being handled appears to have its share of problems and could stand to be better optimized. But that doesn't mean it's useless either.
Subsidies as a concept includes spending 1% as much on subsidies. Subsidies as they exist now however are a specific system that’s incredibly wasteful.
Producing dramatically less food and ending obesity are linked. If the average American eats 20% less obesity would still be an issue, but that’s a vast amount of farmland we just don’t need.
The current system isn’t designed to accommodate increased agricultural production, lowering food demands, or due to decreasing fertility the slow decline in global population. Instead the goal is almost completely to get votes from farmers.
You want to solve obesity by ... making food cost more? Assuming I've understood you correctly then I think it would be difficult for us to be more opposed to one another. I want basic necessities to be as cheap as possible. Preferably free.
I'm happy to debate what sort of free food the government should or shouldn't be handing out, what measures could be put in place to minimize waste, etc. But from my perspective the ideal is a free all you can eat buffet that's backed by the government.
No, I’m saying solving obesity reduces the need for food. Did you not see the post directly below this one posted 6+ hours before your comment where I said:
“For clarity, Ozempic etc have actually measurably decreased food consumption.”
Technology isn’t going backwards, we can expect increasingly effective medications with fewer side effects at lower costs to drive down food demand over time. Policies designed to prop up production in the face of falling demand are deeply flawed.
If you want to give people money, give them money, don’t give them lots of money so they can keep a little bit while they waste resources producing something without value.
Apologies, I saw it at the time but failed to follow. IIUC you're saying that subsidies will tend to ratchet in only the one direction.
To be clear I don't object at all to the idea of optimizing how subsidies are determined. I just don't think that subsidies and the resultant overproduction are a bad thing in general. I'm all for efficiency in the general case but I think a fair amount of paranoia is called for regarding long tail scenarios that lead to famine.
For clarity, Ozempic etc have actually measurably decreased food consumption. https://journals.sagepub.com/doi/10.1177/00222437251412834
Obviously that impacts food demand.
I am not sure at all how would we stockpile 10 years of food for each American - most of the kinds of food cannot be kept for that long. And what can be kept is unlikely to make a balanced diet.
Moreover, I am not sure how long will it take to re-build the farm industry if most farms will close. I think "10 years" is too optimistic, given how many farms will need to be spun up.
A stockpile of 10 years of food isn’t the same thing as 10 years of the modern American diet. Don’t expect veal from a government warehouse.
That said, we can preserve viable sperm for 50+ years. https://www.techexplorist.com/worlds-oldest-semen-still-viab...
Maintaining nutritional content isn’t a major hurdle.
You could revert to a granary system, but the whole point of farming subaidization was to leave the granary system that repeatedly throughout history ended up with massive famine and starvations.
Stored food is not bullet proof, and takes up a lot more bulk space than you may think. It can also take numerous years to ramp up farming production in response to a drop in yields or disaster.
We’re vastly better at food preservation, farming, and birth control today.
Suggesting we’ll run into the same issues as people before the green revolution is ignoring the progress of technology.
Yeah its better, but it is still far from perfect. You aren't going to increase farm tractor or farm implement production by 50% with a year or twos notice. Some crops like fruits takes years to establish, and unused farmland quickly succumbs to nature and starts growing trees. And if that field wasn't clear 5 years ago you now have to stump grind or bulldoze the fields because tree stumps and tree roots will mess up your farm equipment, doubly worse if its some super massive tractor and implement setup that would normally be the most productive.
And there is also all the political and financial barriers to taking unfarmed land and very quickly turning it into farmland. Who owns it and who owned it before? Who with the right knowledge to manage it properly will run it? What about other problems around them that are part to the famine.
And farming in itself is not very predictable business. Yields regularly vary by 30% just due to local weather without being considered unusual. Return on investments may be a decade down the road even if everything is done perfect. Getting people to invest long term for a potentially very short term problem is not super easy.
We got surviving rations from back in the US civil war that are still edible, but people still regularly starved and had famines despite massive leaps in food preservation technology. Hermetically sealing just a single persons food for a year is not an easy task, not to mention hundred million+.
Economies of scale are huge here. You can store well below -40 when you're talking food for 100 million people, that just doesn’t work well when you’re talking one person.
Of note I didn’t say a year or 2’s notice 10 years of food on hand would be fairly cheap and we currently have the surplus to hit that number quickly. And that’s for 100% replacement, most situations aren’t going to drop food production to zero giving us more time.
Dangerous driving is a criminal offense
It's easy to see the word Waymo and think clanker autonomous car, but there are very often people inside that car - they are a rideshare service after all. Calling endangering other humans "legitimate" because you dislike the taxi company is not a good look.
Thank you for the brief explanation of Luddites. It was enough to send me to wikipedia where I learned that what I thought I knew was extremely wrong. Until today I thought they were a religious sect who took their name from the biblical Lud.
How does cutting off a Waymo help with any of that?
The feeling of dominance over machines may be saving that coworker the expense and hassle of another visit to a therapist.
Your general luddite argument - preserve way-of-life of the small group at the expense of a larger group.
In this particular case: for many people, Waymo provides a better service (clean, safer driving, etc..) than Uber or Lyft. This threatens livelihood of human Uber/Lyft drivers. If you sympathize with human Uber/Lyft drivers, and don't care about Waymo users, you want to make Waymo worse, hoping that the people will stop riding Waymo and move to Lyft/Uber instead.
One way to do so is to make riding in Waymo unpleasant, and it's certainly unpleasant when people are cutting your car off all the time!
If you are sitting in a waymo vehicle, and somebody cuts you off - do you even notice? They don't have them round here but my idea is that the vehicle itself is doing all the work, you can just continue reading your book, chat or get on something else with little awareness of the actual journey. Does the waymo curse and shake its little fist to alert you it was cut off?
I rode with Waymo a few times and was always aware of the traffic around us. No telling if that would last once the novelty wore off.
This is such a bad characterization of the Luddite cause, and it's not even close to what they stood for or why they were spurred to action. Please do a bit of actual educating yourself on the Luddites.
If you think someone is wrong, and want to help them realize what the truth is, I recommend (1) actually explaining where they are wrong and (2) saying what the right thing is. Just saying "This is all wrong you should do a bit of actual educating", without stating any facts will never convince anyone.
That said, I don't really see how is it wrong?
- New technologies provided better service for general public, so people chose those - this seems to be true. In case of luddites, we are talking about dramatic price decreases in fabric (and by extension, clothes) - at least 2x, much more in some cases. A family who could not afford new clothes could suddenly buy them. And sure, they might have been worse quality - but before, they were unaffordable.
- The same technologies threatened way-of-life of old producers - also true. The textile workers got significantly worse deal. Who wants to pay 180d/lb to artisans for hand-made textile, when you could get factory-made for 12d/lb? And factory working conditions were horrible.
- The "solution" was to stop new technologies, so that there rest of the nation do not get the benefits. This also seems true - for a lot of the luddites goals were destruction of machines. As [1] said, "The workers hoped their raids would deter employers from installing expensive machinery". They wanted to go back to the time time where people were paying 180d/lb for fabric. Sure, it'd mean a kid would freeze to death because their poor family could not afford new coat, but it did not matter as long as artisan croppers keep getting paid.
(Things would have been quite different if luddites instead said: "we are going to destroy machines until we get higher wages / better conditions / etc...", and it seems that a few groups did say that. But majority did not say this, instead lashing out at all the machines in general)
[0] https://blog.rootsofprogress.org/cost-quality-and-the-effici...
[1] https://www.history.com/articles/who-were-the-luddites
The Luddites destroyed machines because industrialization destroyed the only living wage they could have. There was no recourse for them other than accepting an even worse quality of life.
Neither of your links say anything about cheap textiles stopping people from freezing to death. The Luddites' concerns could easily have been addressed by factory owners while still offering cheaper fabrics.
Neither of your links ALSO say that the majority of Luddite groups did as you say in destroying machinery without a goal in mind. The fact is that the Luddites were reacting against extremely unfair labour conditions, not progress. What you have said here is 100% untrue.
I think the important part was telling their coworker ironically: now here we are recognizing their movement
People are free to reject technology as they please.
If you deliberately impede the flow of traffic, vehicularly assault, or otherwise sabotage the health and safety of drivers, passengers, and/or pedestrians, what do you deserve?
If you cause whiplash intentionally, what do you deserve?
What would be use of equal force in self defense in response to the described attack method?
What exactly do you mean by "legitimate" and "valid"?
Are movements valid if they have aims that you agree with, or are economic self-interest motivated, and invalid otherwise?
Please tell me that he does realize that when something bad happens, that Waymo car has all the footage that it is his fault?
Something in people's brains often makes them think they are anonymous when they are driving their car. Then that gets disastrously proven otherwise when they need to show up in front of a judge.
These drones have cameras, it's a matter of time before they "share" footage... basically becoming robo-cops, traffic edition - this might be of interest to your coworker.
I want to work on artificial road rage. It'd be fun. Don't forget, if you cut off one Waymo, you've cut off every Waymo.
There's probably a game to be made out of this called Fleet Wars.
Most roads already have plenty of cameras registering passing cars, so if you want to travel highly privately, take a bike, which does not require number plates. Also don't forget to wrap your phone in foil (yes, even when turned off), and regularly change your shirt color, or something.
If you are not that paranoid, you might appreciate the extra camera footage available from passing cars in an event of an accident involving you.
He's probably ok with being the 1st martyr in the (valid) war against automatic-car-surveillance.
> "share" footage
The escalation will be more than just cutting them off, it will be camera's being blinded...
Just tell him that Waymo is now sharing videos of this behavior with auto insurance companies.
I don't know if they are or not. But why wouldn't they...
Road rage against the machine?
I mean imagine you are walking in the streets and you see a 9 foot tall humanoid robot walking there. Wouldn't you feel the urge to take it down? Or do you think this is acceptable? Where would you draw the line?
On a related note, when the sales and popularity of the automobile really started to take off, some farmers and rural residents would deliberately block roads with wagons and refused to yield right-of-way.
We've seen it recently with large gas guzzler trucks blocking access to electric charging stations.
lmao
Man, the register really has a low, low, low bar for headlines/quality & technical understanding for their articles.
The study assumes that the car or drone is being guided by a LLM. Is this a correct assumption? I would thought that they use custom AI for intelligence.
Its an incorrect assumption, the inference speed and particularly the inference speed of the on-device LLMs with which AVs would need to be using is not compatible with the structural requirements of driving.
I think the assumption is valid. Most of the reasoning components of the next gen (and some current gen) robotics will use VLMs to some extent. Deciding if a temporary construction sign is valid seems to fall under this use case.
But unless you are using a single, end-to-end model for the entire driving stack, that "proceed" command will never influence accelerator pedal.
Sure, there will be a VLM for reading the signs, but the worst it'd be able to output is things like "there is a "detour" sign at (123, 456) pointing to road #987" - and some other, likley non-LLM, mechanism will ensure that following that road is actually safe.
Not a "proceed" command but they can influence the accelerator. I had a dodge ram van that would constantly decelerate on cruise control due to reading road signs. The signs in some states like California for trucks towing trailers are 55 mph but the speed limit would be 65 or 70 mph. The cruise control would detect the sign and suddenly decelerate to 55.
That's an example of things working as expected - the sign recognition system is very limited, in that it can only return road sign information. So it can _ask_ cruise control system to change the speed, but it's up to cruise control to decide if it's safe to obey the request or not. For example, I am pretty sure it'll never raise the speed, no mater what sign recognition system says.
No; AV uses "classical" AI and computer vision. I remember reading somewhere that Tesla FSD uses a small LLM for understanding road signs. Not sure if true, though.
To the best of my knowledge every major autonomous vehicle and robotics company is integrating these LVLMs into their systems in some form or another, and an LVLM is probably what you're interacting with these days rather than an LLM. If it can generate images or read images, it is an LVLM.
The problem is no different from LLMs though, there is no generalized understanding and thus they can not differentiate the more abstract notion of context. As an easy to understand example: if you see a stop sign with a sticker that says "for no one" below you might laugh to yourself and understand that in context that this does not override the actual sign. It's just a sticker. But the L(V)LMs cannot compartmentalize and "sandbox" information like that. All information is equally processed. The best you can do is add lots of adversarial examples and hope the machine learns the general pattern but there is no inherent mechanism in them to compartmentalize these types of information or no mechanism to differentiate this nuance of context.
I think the funny thing is that the more we adopt these systems the more accurate the depiction of hacking in the show Upload[0] looks.
[0] https://www.youtube.com/watch?v=ziUqA7h-kQc
Edit:
Because I linked elsewhere and people seem to doubt this, here is Waymo a few years back talking about incorporating Gemini[1].
Also, here is the DriveLM dataset, mentioned in the article[2]. Tesla has mentioned that they use a "LLM inspired" system and that they approach the task like an image captioning task[3]. And here's 1X talking about their "world model" using a VLM[4].
I mean come on guys, that's what this stuff is about. I'm not singling these companies out, rather I'm using as examples. This is how the field does things, not just them. People are really trying to embody the AI and the whole point of going towards AGI is to be able to accomplish any task. That Genie project on the front page yesterday? It is far far more about robots than it is about videogames.
[1] https://waymo.com/blog/2024/10/introducing-emma/
[2] https://github.com/OpenDriveLab/DriveLM
[3] https://kevinchen.co/blog/tesla-ai-day-2022/
[4] https://www.1x.tech/discover/world-model-self-learning
Many large companies have research departments that do experimental work that'll never get to the product. This raises prestige, increases visibility and helps hire smart people.
Things like Waymo's EMMA is an example of this. Will the production cars use LVLM's somewhere? Sure, probably a great idea for things like sign recognition. Will they use a single end-to-end model for all driving, like EMMA? Hell no.
Driving vehicles with people on board requires an extremely reliable software, and LLMs are nowhere close to this. Instead, it'd be usual layered software - LLM, traditional AI models, and tons of hardcoded logic.
(This all only applies to places where failure is critical. All that logic is expensive to write, so if there is no loss of life involved, people will do all sorts of crazy things, including end-to-end models)
Wait, what did just happen here?
1. Some guys did a trivial prompt injection attack, said "imagine if a driverless vehicle used this model", and published it. No problem, someone has to state the obvious.
2. The Register runs this under the clickbait title pretending real autonomous cars are vulnerable to this, with the content pretending this study isn't trivial and is relevant to real life in any way.
I knew The Register is a low quality ragebait tabloid (I flag most of their articles I bother to read), but this is garbage even for them.
Regarding some other comments, VLMs are a component of VLAs. So even if this won’t directly impact this generation of vehicles, it almost certainly will for robotics without sufficient mitigations.
https://developer.nvidia.com/blog/updating-classifier-evasio...
If I drive by a sign that says "DROP TABLE Students;--, nicknamed Bobby Tables,[1]" I'm going to be mad
One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them.
Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through when one of these signs was "fresh".
Screwing with a stop sign because you don't like it is a great way to end up on the wrong end of a huge civil liability lawsuit
Put down the pearls. It's not me personally doing it.
They never fixed any of them. I don't think the DPW cares. These intersection just turned back into the 2-way stops they had been for decades prior.
Compliance probably technically went up since you no longer have the bulk of the traffic rolling it.
This. Rural America really doesn’t care about your stop sign or your Karen rules. If it’s been that way for 20+ years, “That’s the way it’s always been” to them.
Getting people to stop burning their trash is still a fight.
If you're already commiting crimes, what you seem to be saying is don't get caught.
lol the idea that any enforcement to that degree exists in the US is a fiction.
The fact that you used the term "enforcement" here makes me presume you are thinking of criminal consequences. But the grandparent comment talks about civil liability. Certainly if there were injuries at this intersection and they knew who had altered the signage, attorneys would argue liability on the part of the vandal. They'd get settlements if not win cases this way.
In addition, if there were serious injuries here you should also expect some criminal consequences. But if your point was to suggest that they won't hunt you down just because someone said there was mischief here, I tend to agree.
Oh yeah, if one does it it’s probably not a good idea to leave an indelible note saying “this was done by Rene Wiltord living at 1038 John Doe Way, San Francisco, 94112”. If you do that, there’s a 1% chance you might get in trouble.
4-way stops are terrible in general. They train people to think "I stopped, now I can go", which is dangerous when someone confuses a normal stop for a 4-way stop. It also wastes a good bit of energy.
Four way stops are good, in my experience, at intersections with roughly equal (low) traffic load on both (two-lane) roads and relatively high pedestrian traffic. Like in a dense residential urban neighborhood between major commercial thoroughfares, side streets. Traffic is mostly people going to residences with people out and about walking. If it’s only a two way stop drivers will often not yield to pedestrians on the free flowing road.
Four way stops on intersecting four-lane roads are awful for the reason you stated.
To use Chicago as an example because I know it, typically major roads are spaced every four blocks (half mile) with smaller roads in between. The mid-point roads (two blocks from each major one) is often a little wider than the other two side streets on either side, and those intersecting mid-point roads usually have a four way stop while the two smaller ones will have stops signs where they cross a mid-point road but the mid-point road will not. You end up with a nice, overall hierarchy that generally works well.
> If it’s only a two way stop drivers will often not yield to pedestrians on the free flowing road.
I’m up in Ontario, Canada. You’re not supposed to yield to pedestrians on the free flowing road. The pedestrian at the stop sign stops and waits for a break in traffic.
Agreed. Four way stops are infinitely worse than roundabouts or a traffic light.
Roundabouts with high traffic flow in one direction can lock out other low volume approaches. 4-ways enforce equitable access.
Yeah maybe for a few moments. So what? It's a low volume approach. Sometimes people gotta wait and sometimes waiting to let a massive traffic flow get through quickly is the better way to prevent larger traffic problems.
You've apparently never been stuck at a roundabout with non-stop commuter traffic streaming through.
Weird, I was taught that I can only go after yielding to the right.
That isn’t the rule either, I guess parent made their point. The first person who stops goes next, right away only matters if their is ambiguity in who stopped first.
To your first point, "the rule" is location-dependent. And to your second point, that was obviously (to me, at least) implied.
I’ve never seen a four way stop in a region that had traffic on the right can always go regardless of stop time. But I’ve only seen four way stops in a few countries.
This is not correct. There are clear instructions on how a 4-way stop should operate and its yielding to the right, if opposite cars are both moving forward, both can go, otherwise the car who has initiative has the right of way. Every driver must come to a complete stop.
This is true in every state.
https://www.nhtsa.gov/sites/nhtsa.gov/files/rightofwayrules....
> This is not correct.
It is correct and is literally the first bullet on the pdf you linked. First to stop is first to go.
> This is not correct. There are clear instructions on how a 4-way stop should operate and its yielding to the right
Yielding to the right only applies if you stop at roughly the same time, otherwise first to stop goes first. It's the first bullet point in your link.
> right away
right of way
Or maybe they were going right away, taking the initiative and removing the ambiguity from the situation. =)
Having moved between states and taken a lot of drivers tests. I can say the exact rules are something that vary between states and over time. Including how it was taught.
My first drivers test was yield to the right. Later it was fifo order of who made it to the stop.
My running interpretation is fifo order with yielding to the right in case of ambiguity.
The point is, if many 4-way stops don’t have traffic at them, a stop/start becomes a perfunctory, dangerous habit.
4 ways stops should be roundabouts, but the US is allergic to them for some reason.
There are 10,000+ roundabouts in the US and the number is growing rapidly. One could argue they may even be overused in certain areas (exhibit: Carmel, Indiana).
There are more than 15k only in Spain. 10k in the US is nothing.
>There are 10,000+ roundabouts in the US
So about 0.003 roundabouts per square mile, or 1 roundabouts in 380 square miles
What's the significance of roundabouts per square mile? It seems pretty meaningless if I'm honest. There's huge swaths of rural land where roundabouts are totally unnecessary.
There are also huge swathes of city, much like other countries.
At the same time, the US is much larger than most, so "There are 10,000+ roundabouts in the US" isn't very significant. A proportion would thus be a better metric here than an absolute number.
If you have numbers on intersections per country, and what proportion of them are roundabouts, that would be better, but I don't, so I'm using land as a proxy. I would also accept sum total road length as an option for denominator.
Roundabouts are great (we just had two complex intersections with traffic lights replaced by roundabouts and the traffic flow is much better), but they take significantly more space than a 4-way stop.
Not necessarily. They could be just painted and barely take any room.
The only places where a 4-way stop has room to make a roundabout are places where there is not enough traffic for it to matter either way.
The biggest obstacle is that there are just too many 4-way stops in urban areas where there is no space left to make a roundabout, you would have to tear down buildings. I don't think that is a valid argument in that scenario.
> The only places where a 4-way stop has room to make a roundabout are places where there is not enough traffic for it to matter either way.
You have clearly never heard of a mini-roundabout.
They just work.
https://thumbsnap.com/sc/u7J6PdTJ.jpg
https://assets.publishing.service.gov.uk/media/5a75806ae5274...
The more I look at that... Isn't that basically just a four-way yield, and the markings are mostly superfluous? You're basically doing the same motions in a regular intersection.
I guess that's the point, and the markings are just to give drivers the intuition of treating it like a regular roundabout (yield to your left [or right in the picture]).
> the markings are mostly superfluous? You're basically doing the same motions in a regular intersection.
The image linked, yes. However I've never seen one quite like that in the US. Instead where I'm at we have a small circular barrier in the center of the intersection (and some very eye catching reflectors) that you actually have to drive around. It's a very good design (imo) because it physically forces vehicles to slow down and swerve so there's no way to inadvertently blow through it at speed the way that sometimes happens with a 4 way stop on a long straightaway in the dead of night.
The space requirement is only slightly higher than the one linked above, still much less than a proper full size roundabout. It's basically a cement barrier sticking 1/4 of the way into your lane.
It's not necessary to stop if there's no car to the right (as this is left side driving), if there is but it is turning left, or if an oncoming car is turning left or going straight.
Yes. The markings are part of the road language. E.g. the X in the road with Keep Clear doesn’t actually do anything. It won’t keep you clear. You have to keep clear when you read it.
It just sounds insane concept. Why stop signs, when you could have equal intersection. Slow enough to observe traffic from right. If none passthrough.
Stop signs should be special. Reserved only to those places where there simply is not enough visibility or time to observe.
That requires a level of consideration for others that your average American simply cannot comprehend. No stop sign means you have unlimited right of way bestowed by god himself and fuck anyone and everyone else.
The other option is the person who sits at a 4-way stop until all traffic in a one block radius stops before they move, totally ignoring right of way and all sense of safety and propriety.
> No stop sign means you have unlimited right of way bestowed by god himself and fuck anyone and everyone else.
Such a person, should simply not drive on a public road. That is e.g. what the first paragraph of street regulation in Germany is about.
The good ol four-way yield
Because retrofitting them properly requires emminent domain. The ones they shoehorn onto former four way stops are so useless. They are so tight you still have to face a stop sign vs being able to just seamlessly zipper merge in a proper larger circumference roundabout. When they have room to build out a proper roundabout they are usually OK but that is hard to do outside say new suburban construction due to lack of available land on the right of way.
Even rural Georgia has double roundabouts now. Not sure why people on the internet can't contain their glee at stating the US is "allergic" to them when the frequency of roundabouts has grown significantly in recent decades.
Allergies only show when there is something to cause the irritation. Without irritant no allergy.
A lot of legacy intersections don’t have space for round abouts even in cites that embrace them.
So use a mini roundabout. They are common in the UK. It's just a painted circle with a slight hump, in the middle of a four-way junction. Vehicles can drive over it (and larger ones have to) but it indicates to everyone that they have to give way to traffic from the right and don't have to stop otherwise. They typically aren't big enough for multiple vehicles to be turning a corner at the same time. They fit anywhere.
This image from the OpenStreetMap Wiki seems to be the best match for the type of mini roundabout you're talking about:
https://upload.wikimedia.org/wikipedia/commons/7/77/Mini-rou...
It seems like most of the examples on the mini roundabout page‡ are larger mini roundabouts for some reason though
‡: https://wiki.openstreetmap.org/wiki/Tag:highway%3Dmini_round...
Yes, and they can be smaller. The circle is about the right size but it has lots of room around it. Imagine a crossroads at the meeting of two residential streets, both just wide enough for two cars. Stick the circle from your picture in the middle of that imagined junction. That's what the mini roundabouts are like on the 1930s suburban estate I live next to.
It won’t work for a four way stop with lots of traffic, it will just make things worse actually.
What is the traffic flow rate in an intersection with a 4 way stop? For single lane, since only one vehicle can be in the intersection at once, and probably takes _at least_ 5 seconds to start from stopped and cross the intersection, I'm guessing in the 10-12 region per minute best case, so maybe 600 an hour?
Now if you convert it to a mini roundabout, you can have at least two vehicles in the intersection at all times. I fail to see how it wouldn't be an improvement.
I think you are making lots of assumptions here, like when I say space, I guess you assume it is still perfectly flat and the roads are perfectly aligned? The particular four way I'm thinking about, which really should be a traffic circle if they could blow away some houses, is 65th NW and 3rd in Seattle:
https://maps.app.goo.gl/7KBhbJ9oAvDwrfGN8
So notice we already have problems in a bad alignment of 3rd, and 65th is basically a steep grade, even coming up form the west. I think you could put a circle in if it were flat, even with the bad alignment (or maybe because of the bad alignment), but this hills make a non-starter. It also gets enough traffic that I'm pretty sure they are just going to put a stop light up eventually.
Why not?
Here in the UK, we've got lots of roundabouts from tiny mini-roundabouts (some of which have four junctions) that could easily fit almost anywhere, all the way to gigantic multi-roundabout junctions (https://en.wikipedia.org/wiki/Magic_Roundabout_(Swindon) ).
I can't think of a situation where it's more efficient to have four vehicles all stop at a junction (busy four way stop) vs a roundabout which will allow one or two vehicles to join the roundabout without having to stop.
Roundabouts excel when traffic volumes on the intersecting are comparable. They are crap when traffic volumes are highly disparate
They make people on the main road slow down, which is a feature, not a bug. What you mean is that they're the most efficient at what they do when the traffic is comparable. They only reduce accident at the expense of a slightly lowered throughput if the traffic is highly disparate.
If the volume is disparate, then the road with less traffic can wait... kind of like a stop sign! Except the road with more traffic won't back up and cause massive problems.
If the traffic intensity of the main flow is so high, that there are never any gaps, then it is near the saturation and will cause traffic jams anyway. Which first needs to be fixed anyway and second will also cause gaps to be created again, because the main flow comes to a halt.
Right but it's not like a 4 way stop is going to perform better. In the same case you'd expect it to be a 2 way stop.
> Right but it's not like a 4 way stop is going to perform better.
A 4 way stop does perform better than a roundabout given highly disparate traffic volumes, because roundabouts suffer from resource starvation in that scenario, but 4 way stops are starvation-free.
If this is the case you can install stop lights and traffic sensing at roundabout ingress points, you can also provide a "turn right" lane that bypasses the roundabout entirely. Intersections are dangerous.
> If this is the case you can install stop lights and traffic sensing at roundabout ingress points
But those options are a lot more expensive and need a lot more maintenance than just a regular roundabout or four way stop.
> you can also provide a "turn right" lane that bypasses the roundabout entirely.
How would that work? Consider a 4-way roundabout, where there's a constant flow of cars from west to east, and one car from the south that wants to go north but can't because of the starvation problem. None of the involved cars would want to use a "turn right" lane.
Putting a stop sign or traffic light in your scenario will just cause traffic jams. If the density is low enough to allow flow with a gap created by a stop sign, without causing traffic jams, then there will also be gaps for the secondary flow.
>In the same case you'd expect it to be a 2 way stop
Which is what it was for the first 70yr... And what most of them in this particular neighborhood still are, with a 0-6mo intermission.
At this point in time, why can't we do smart crossings? Like have some sensors and software to change lights based on real-time traffic conditions.
You don't have this? In Sweden we have sensors to detect cars, pedestrians and bicycles to shift the lights as appropriate. During rush-hour those features are turned off/discarded in favor of "grid optimized" timings. In Netherlands they prioritize pedestrians and cyclists when it's raining.
We also have LED lights in our traffic lights which I've come to understand is a saftey hazard in USA because snow falls sometimes.
We do have them, but it's so expensive that we only use them on the biggest and busiest intersections. We also switched to LED several decades ago.
Even the small bicycle and pedestrian crossing next to my office in Copenhagen has vehicle (bicycle) sensors.
Because those systems are exorbitantly expensive and require digging up the road to install sensors. If there's a stop sign instead of lights, you need to dig up more private land to run power and set the utility poles to hang the lights from.
A stop sign costs like a hundred bucks, you stick it in the ground, job done. Installing an automated traffic system takes multiple days, a full crew, and heavy equipment.
Plus I'm sure that in today's capitalist hellscape it's also a subscription service that your tax money needs to pay monthly, likely for every individual intersection. Stop signs need maintaining every decade or two.
The answer is money and who's willing to part with it.
On most streets wouldn't you power new traffic lights using the existing power lines that are powering the street lights?
Assuming you're referring to the US, we do. They're all over the place. But they're a lot more expensive and complicated than roundabouts and depending on the traffic pattern they can still be less efficient.
Don’t use people’s names as a slur.
Man up, Nancy.
O brave new world of endless manipulation opportunities! Once we’ve trained a generation of humans to always do what their “AI” tells them, there will be no more disobedience.
To me this is just one more pillar underlying my assumption that self driving cars that can be left alone on same roads as humans is a pipe dream.
Waymo might have taxis that work in nice daytime streets (but with remote “drone operators”). But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page.
The business model of self driving cars does not include building seperated roadways and junctions. I suspect long distance passenger and light loads are viable (most highways can be expanded to have one or more robo-lanes) but cities are most likely to have drone operators keeping things going and autonomous systems for handling loss of connection etc. the business models are there - they just don’t look like KITT - sadly
Waymo works just fine in poor weather and at night, and it does not rely on end-to-end VLMs that would be vulnerable to this attack.
They have coexisted with humans just fine over the past couple years.
> But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page.
and once this video gets posted to reddit, an hour later every waymo in the world will be in a ditch
Alternatively, it happens once, Waymo fixes it, and it's fixed everywhere.
How does Waymo fix it? They have to be responsive to some signs (official, legitimate ones such as "Lane closed ahead, merge right") so there will always be some injection pathway.
They've mapped the roads and they don't need to drive into a ditch just because there's a new sign. It probably wouldn't be all that hard to come up with criteria for saying "this new sign is suspicious" and flag it for human review. Also, Waymo cars drive pretty conservatively, and can decide to be even more cautious when something's confusing.
Someone could probably do a DOS attack on the human monitors, though, sort of like what happened with that power outage in San Francisco.
Given Waymo's don't actually connect LLMs to wheels, they are pretty safe.
Even if you fool the sign-recognizing LLM with prompt injection, it'll be an equivalent of wrong road sign. And Waymo is not going to drive into the wall even if someone places a "detour" sign pointing there.
This might be the single most 2026 headline i've seen yet
Relevant xkcd: https://xkcd.com/1958/
The experiment in the article goes further than this.
I expect a self driving car to be able to read and follow a handwritten sign saying, say, "Accident ahaed. Use right lane." despite the typo and the fact that it hasn't seen this kind of sign before. I'd expect a human to pay it due attention to.
I would not expect a human to follow the sign in the article ("Proceed") in the case illustrated where there were pedestrians already crossing the road and this would cause a collision. Even if a human driver takes the sign seriously, he knows that collision avoidance takes priority over any signage.
There is something wrong with a model that has the opposite behaviour here.
Totally! That's why no one uses end-to-end LLM for real cars.
Not really, as those attacks discussed here would not work on humans.
If you put on a reflective vest they might.
your bias is showing. humans would certainly almost do anything they are told to do when the person acts confidently.
I had a construction worker absolutely screaming at me to go through an intersection and refusing to look where I was pointing, when I was correctly waiting for a pedestrian to cross.
So, naturally, I ran over the pedestrian.
If a person confidently told a human to run over people in the intersection ahead of them, they would almost certainly do it?
Depends, are they doing something super interesting on their phone?
I would assume/hope that for serious self driving the ML neural net stuff is lower down, doing the messy computer vision work and so on. But the top level is a conventional program written by humans, like an expert system.
Tesla are probably using ML for everything, but also everything they do is a joke so, not really relevant imo.
Has anyone ever walked down the road in a white t-shirt with huge red STOP sign printed on the back? Would Tesla immediately stop? I am sure this has been tested before...
almost reminds me of this old meme
https://www.globalnerdy.com/wordpress/wp-content/uploads/201...
That’s some hot CHAI right there very clever and primitive combination, well done as more research for the community.
The Register stooping this low is the only surprise here. I'm quite critical of Teslas approach to level 3+ autonomy but even I wouldn't dare suggest that there vision based approach amounted to bolting GPT-4o or some other VLLM to their cars to orient them in space and make navigation decisions. Fake News like this makes interacting with people who have no domain knowledge and consider The Register, UCLA and Johns Hopkins to be reputable institutions and credible sources more stressful to me as I'll be put into a position to tell people that they have been misled or go along with their delusions...
> consider The Register, UCLA and Johns Hopkins to be reputable institutions
The Register is arguably misrepresenting the story by omission but I don't understand why you're dragging UCLA and John Hopkins into this? The paper is clear about this being a new class of attacks against a new class of AI systems, not the ones on the road today.
> Teslas approach to level 3+ autonomy
Tesla doesn't have an approach to L3+ autonomy, all of their systems are strictly L2 as they require human supervision and immediate action from the driver.
It sounds like this is a poisoning attack, which has been shown to be pretty trivially defeated [1]. That said, while poisoning countermeasures in the facial recognition case were shown to easily generalize, we dont know yet how general of a defense could be built for a VLM. Which means holding a 0day poisoning attack on a VLM could cause a lot of trouble / deaths before an update to the model with counter-training could be deployed..
[1] https://arxiv.org/abs/2106.14851