I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.
At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.
It's already been happening for a very long time. The US' "Operation Earnest Voice" happened in 2010/2011[0]
And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]
> Indeed, seeing how slanted online discussion was, I suspect one side had many more bots, and I'm not sure if the other side had any.
I think your bias is showing. From what I recall, both sides employed the exact same tactics, and we often saw blatantly inorganic discussions popping up in random subreddits on how $(SIDE_A) was the real villain and $(SIDE_B) only had supports because they were paid bots driven by their PR firm.
Things reached a point where I'm sure the same template was used by each counterpart.
On a bigger picture, I'm not sure why anyone still goes to places like reddit and believes anything showing up in the mainpage is still organic.
Is there any writing outside of your recollection that says the Heard side was using bots? Afaik all reporting suggested it was one sided, and not her side
this has been happening since the dawn of information-communication systems basically. because ever since, people have had the ability to listen in, copy, learn from etc. - so there's always been incentive to spread disinformation for some parties.
brand defence,
nation-state influence operations.
There are many different flavors of the same thing, trying to mislead competitors or enemies. Or to dupe consumers ofc... Since the tech is more and more available, now also 12 year olds can run insanely large influence campaigns, to boost their streaming profiles or other somewhat benign things, but obviously modelled after what larger players have been doing forever.
The first transatlantic commercial transatlantic phoneline (TAT-1) was immediately tapped on both sides by UK/US... before that it went over radio, which was simply intercepted.
Because of that nature of the coms system being easily listened in on, just like they do/did with the lil mirror-beam-splitters at GCHQ etc.., its been natural effect to then provide disinformation over those channels. perhaps even use those channels exclusively for disinformation, and have a secure channel to do _actual_ coms.
So as you can imagine there has been many decades now of practice in why and how to share disinformation effectively to undermine other peoples work or processes.
All threads that have... certain keywords... are like this. Talk about meta. Looks like I'm now in the positive (+5) after being in the negative just a moment ago.
Anyways, I got really interested in this topic around mid 2010s because of some really interesting reporting by Codastory[0] covering Russian "troll factories" especially used to influence opinion in eastern Ukraine, but also in many other circumstances. In Houston in 2016, Russian troll accounts planned both an ANTI Muslim rally AND a counterprotest to that rally all on Facebook.[1] Hundreds of real human beings showed up!
Russia got a ton of attention for their use of troll farms but the truth is that the US and certain allies have actually been using stuff like this for far longer and, imo, with much more success. In fact, Coda Story itself is funded by the NED which is a massive pillar of global US, uh, "soft power".
For me the main difference is if the activity is clandestine and meant to deceive/hide the origin. Everybody tries to influence things their way. Doing it covertly is a hostile act. That does not mean that I would not condone hostile acts against my/our enemies but it is something take into account. And regarding your Russian examples, it showed for how long they have seen the west as enemies and conducted hostile acts.
They view certain components of "The West" as their enemy - indeed, the very organizations which conduct the same operations against the Russian people.
Because cognitive biases are one of the hardest things to even acknowledge, let alone suppress. All the media out there has such a strong bubble effect, because people hate being challenged and love to stay in their comfy echo chambers. Remember labubu? Bao buns certainly do not.
There have been so many bot armies exposed over the years that one would think every narrative being pushed should be considered to be backed by bots. Some state-level actors even operate their narrative-pushing campaigns openly, yet people still happily deny their entire existence.
HN is very much not an exception to this rule. It's very hard to accept that maybe some thoughts you have were actually installed.
Maybe a few are just “people” (i.e. the same people who normally post in the community.)
But in most cases I’d suspect any such botting campaign brings along with it a few human agitprop agents embedded onto each target platform, well-positioned to deny and discredit anyone pushing back on the narrative the bot farm is spreading.
(Sometimes this is pretty obvious — a lot of accounts coming out of the woodwork to deny Chinese bot farms have a conspicuous “English as learned as a second language in Beijing, to a competent but not fluent level” writing style that does not often otherwise crop up in other discussions in those same communities. But other times it can be far more subtle, making use of operatives hired from within the community’s own majority-culture rather than relying on foreign nationals playing the part of same.)
There’s bots that automatically downvotes certain topics on HN. I post at times that often coincide with the middle of the night in North America, and I get 3–5 downvotes within the next 5 minutes.
Talking about a country that sets up fake think tanks is a good way of triggering them.
It's already been everywhere for years. The most obvious one is any time someone says something bad about Russia or China, a load of comments pop up to say "Yeah but what about when america does <x>?". Never actually attempting to discredit the statement, just to steer it towards the US
The sheer volume of comments make it super obvious when it happens compared to normal conversation
Propaganda is not a new thing. It might look slightly different on the internet, but it doesn't look that different. Creating organizations to push a narrative was just as much a thing 100 years ago as it is now.
The whole hysteria about 'fake news' 10 years ago was apparently an outrage against the public being duped, whereas is was in fact frustration that spreading fake news has been democratised.
> Throughout Europe in the 16th century, printed tracts were used to argue religious doctrine and foment support for religious causes. In England, Henry VIII used print literature to justify his break from the Catholic Church. During the subsequent reigns of Edward and Mary, print polemics escalated into propaganda warfare, as print media gained enormous potential to sway common opinion.
> Over the next century, the pamphlet became the principal means of garnering support for a cause or an idea, and was particularly influential during the English Civil Wars (1642-1651) and the Glorious Revolution of 1688.
But this also means there’s more competition for attention, because now all lunatics can post from their bedrooms. The institutional media (including legacy) still maintain strong presence.
Of course, a great deal of panic over fake news wasn’t necessarily over fake news per se, but whose fake news and the loss of control over the “official narrative” that legacy media used to have a monopoly over. This isn’t a vindication of the heaps of slop published on the internet, just a more synoptic take IMO.
People miss Cronkite, because that style of media made them feel certain in their opinions and in “official consensus reality”, regardless of whether it was true or not and whether it was a trustworthy authority or not.
Scale is different though. Tapping a phone isn't new, tapping _all phones_ is different. Altering a photo to change what it appears to depict isn't new, being able to do it in a few seconds and for free is different. So sure, propaganda isn't new, but being able to pump out an organization's worth of credible-looking content for cheap absolutely is.
IIRC the first or second chapter of Bernays' Propaganda (1928) talks about sophisticated influence campaigns in the fashion industry. Conscripting "influencers" to induce demand, and so on. As you said, literally 100 years ago.
Jill Fields at Cal State Fresno have actually narrowed it down to a (temporarily successful, but not in the long term, obviously) effort by the corset-making industry seeking to stay relevant after WWI during which corsets basically disappeared and there was an organized campaign that was organized both in person and through trade magazines. Women's silhouettes basically took a quick 180 right between 1900 and the start of the war only to turn very very retro in 1919 and while the first was organic and resembled how styles change in the century prior, albeit much quicker, the 1919 switch was openly plotted out in writing that was published but in such a niche publication that I get the sense that people not in the industry didn't even know that it existed. The cite to her paper on Google Scholar: https://scholar.google.com/scholar?cluster=17641417785638413...
You might have to shadow library this one, but it's a good read. Interestingly it seemed to be something entirely separate from Bernays whose observations were based on a derivation of what began as a homebrewed effort in a corner of the market. Valerie Steele wrote extensively about the trends and the after life but the source seems to originate in corsetry and ended up being copied by the garmet industry generally in the 20s.
you can still find books published within last 50 years, you don't have to read books published this year, unless your library went through destruction of "old" books which are now more relevant than ever...
Of course they are. It’s everywhere, especially thr online retailers these days. [1]
Thankfully the resources at our local libraries are a) selected by a combination of librarian curation and public demand, b) often published before 2022, c) heavy things printed on paper or stored in a database and unlikely to suddenly change their content or disappear overnight when a new news cycle breaks.
Written works aren’t necessarily credible even if they aren’t slop, but thanks to curators who care, our libraries are significantly less susceptible (although definitely not immune [2]) to ai slop, and propaganda and manipulation efforts than a search engine. The internet continues to enshittify, and truth becomes further eroded by this coming wave of ai enabled propaganda, we are going to have to turn somewhere if we want to find trustworthy resources. Personally, if it gets to the point that verifying sources from public search results becomes too hard with all the new misinformation and slop, I’ll start propritizing human curation over SEO and start visiting my local library’s website a lot more.
Right; the same mechanisms (curation, demand-driven economies of scale, library user demand) that largely stop niche self-published books from ever finding themselves in libraries (well, nationally in public libraries at least; they are often found in individual branches) work against pure slop books ever turning up.
Books where the author has successfully hoodwinked the publishing process into believing that AI generated text is their own, those will turn up, just as plagiarised books will.
Some nuances here. There’s a variety of different roadblocks at different levels that will work to prevent pure AI slop books ever turning up in a typical public library of physical printed works; I think they never will.
Probably a handful of books in such libraries already have some text in them where the author has tried to pass off fragments of LLM-generated prose as their own and have got that past editors, reviewers and publishers, and that will sadly get worse.
I know my library system at least won't be purchasing AI books. They have a pretty rigorous acquisition process for books and other media thanks to the "concern groups" who sprang up complaining about themes and content they didn't like in libraries. We did have in my state a rural school district figured out recently that their Texas-based reading and instructional materials are AI-slop but they at least figured it out before the school year began. It was unclear how these materials were approved but it apparently bypassed normal curricula approval.
It's so obvious when a book is generated by an LLM that I would blame the person buying the book moreso than I would the person who generated it for it. You can tell in a few sentences if something is AI generated (and by proxy - if it's worth reading or not).
I think using LLMs to research is completely fine, as long as authors would take the information and synthesize their own opinions based on the findings. I read books/articles/posts/comments to hear other PEOPLE's opinions, not LLM generated opinions.
I'm kind of hopeful AI some wikipedia-inspired AI "oracle" will happen in the future. Why not use some central AI to separate fact from fiction (faster than trolls and griefers can come to muddy the debate).
This is based on the assumption of facts existing.
There are many studies, but each can be wrong and they can collectively show a bias. Even things of which are the most non-political of facts can have very strong biases. Look at the Millikan measurement of the electron and how in created confirmation bias and an anchoring effect on some property that has absolutely no real world significance to the things people are tribalistic about (aka, no political relevance). Now imagine the same applied to fields like economics or psychology which do have massive legal/political implications.
For a different example, ask the question if X committed crime Y. There are cases where they weren't found guilty but it is reasonable to assume they did. But being found guilty doesn't make it a fact either, as some people are wrongly convicted. Some eventually are overturned, but even if it isn't, it still isn't a fact they committed a crime.
Then there is the simple ambiguity of statements. Language generally can't support facts. It is why legalize, and programming code, and math's are effectively their own languages. For a simple example, consider the Betrand paradox(1).
>Consider an equilateral triangle that is inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?
Is the answer 1/2, 1/3, or 1/4? Well, it is all three at once, depending upon what you meant by random. Now, imagine how this impacts things like research studies, where the randomness is much harder to quantify and there is constant pressure to p hack a result.
I mean, a big RAG setup with a smart LLM ought to do it, right? Or even just locally provided knowledge database - given the size of LLMs, what's a clone of wikipedia and whatever else you'd need?
it's happening for years now, just in slightly different form, so far it was called disinformation, seo, astroturfing, hasb... ehem... last two years were devastating for charred remains of twitter with all those llm bots having free reign, twitter went from global news board to global gutter in record time thanks to disinformation and astroturfing campaigns
> At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
Whining about AI is a red herring. If anything, LLMs lowered the cost of producing this type of propaganda.
What's novel about this approach is that the emphasis on building up a facade of authority and credibility is not a central part of the propaganda effort. They can get away with outputting the content in a way thats ingested by other LLMs among the reputable sources, and then they lean upon people's gullibility towards LLM output as some sort of credibility laundering.
There is a reason why one of Wikipedia's central policy is requiring reputable primary sources to support content.
Although comments below point out that this has been going on for a long time, I too think it will be geetting infinitely worse in the near future. Everyone will want their own poisoning factory.
One consequence is that one should be extra-wary when asking AI about current, controversial topics.
But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?
That could also backfire. If all information on the net becomes unreliable - you can't judge by quantity, or by authority anymore - then a useful model has to answer from first principles. Train it on simple texts alone so it learns language and reasoning, feed it an ontology, and let it come up with some absolutely unexpected results.
"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."
I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.
That's tomorrow's problem. Losing the PR war is an existential crisis for Israel, so they'll try to "patch the leak" by any means necessary before the boat sinks.
This is new? There were tons of fake news sites, fake blogs, and sock puppet and troll farm accounts before AI.
I guess the new angle might be that AI scrapers have even less discernment than humans, so you don’t have to put much effort into creating a fake front online to sell it. Humans have at least a little bit of radar for this.
… which means the AI companies now have to run input through existing models to audit it before training on it? That’s interesting.
> AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information
Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.
Sama saw this coming a decade away, which is why he invested in Worldcoin (now World). At some point people (and agents) will be clamoring for human verification at the speed of technology.
I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.
I think we need a similar technology to World being built by a more trustworthy company. The question is how trustworthy does one need to be for people to trust you with their very online identities that are literally needed for others to recognize them as human?
Mostly because they already are responsible for systematizing documents that are needed to recognize us as citizens, land or car owners, etc.
Some laid back countries might allow companies to take and manage that function, but I don't think most of them would. Especially not in current landscape where people heard "you are the product" from all tech companies for the last 10 or so years.
It's legitimately weird to see anybody defend them, as the immorality is out in the open.
The past few years have really opened my eyes to the nakedly tribal nature of democratic politics. Polybius was largely correct when he said that it inevitably degenerates into oligarchy or mob-rule -- but what's especially unfortunate is that the US and Israel appear to be getting both at the same time: A naked oligarchy with just enough mob rule to keep the animal spirits of the populace sated.
When such statements/actions are attributed to Ben-Gvir, I assume they and the objective nuanced truth are basically the same. I find it wild how people who are not like him defend him.
Trump just recently said he’s considering attacking Oman.
Politicians say really terrible, stupid things on a fairly regular basis. The difference between the two situations is that Ben-Gvir is loathed by 90% of Israeli society (Trump still has 33% approval) and unlike Trump has no power to follow through on his threats.[1]
Yes, he is totally powerless, he merely controls the police and prison system. (BTW, the Israeli government just announced it plans to transfer law enforcement in the West Bank over to Ben-Gvir's police: https://www.theguardian.com/world/2026/aug/14/israeli-milita...)
It’s an injunction and arguably a legal pretext to stop it in its tracks. That’s not uncommon in any system, just like SCOTUS rulings often seem strangely specific to restrain the scope of their ruling.
Tl;dr Lawfare is a thing and it’s effing everywhere.
I have a long-held feeling with Trump that his most ardent haters don't hate his specific policies, merely that he's so gauche about it, that he has no class.
I have no doubt that Ben-Gvir has a similar segment of haters in Israel. They want the same things, but they don't want to do it in the style of a leering hooligan fishing for a fistfight.
From what I can see, most people hate trump as a solid bloc. The fact that he, as every populist everywhere in past centuries, pointed out to some real problems but didn't do a squat about it when he came to power and in his case massively worsened the situation, is hard to escape from. He certainly didn't rise above other populists in this.
His form is just a tiny cherry on top of very big rotting cake. Make form smooth and underlying crap won't change a bit. Hating trump ain't emotional, purely rational approach is sufficient.
Definitively not a solid block. It's tempting to believe that, but no, John Bolton probably hates quite different things about Trump from you (and definitively from me). A lot of the opposition both outside the republican party and inside (such as it is), is probably entirely fine with bombing Iran for instance. Not mad he's bombing it, mad he's bungling it.
> I have a long-held feeling with Trump that his most ardent haters don't hate his specific policies, merely that he's so gauche about it, that he has no class.
A bit of an extraordinary belief to hold, to be honest
Like Islamic terrorist organizations, dictatorships, democracies et al have never tried to influence anything either, my point isn't that you're wrong you just sound like a fool trying to pretend this is some "Israeli only" thing. Which is totally in vogue now.
Not that relevant here, as they try to attack Israel with that influence. Like Hamas that organize some protests in the EU [1] or Qatar's Al-Jazeera censoring anti-Hamas criticism from Gazans [2]. So it's more like "if they are attacked by that? why can't they use it themselves."
Like, shooting a gun is bad. But shooting the shooter? Not the same (thus, not "whataboutism" because you can't say "who care about other shooters in the scene - whataboutism!") .
Shooting the shooter? Are you suggesting that all of the tens of thousands of children killed in gaza were shooters? What I don't understand about these types of arguments is that it's not clear who you are trying to convince. Those who already agree with you don't need convincing, and the rest of us aren't going to buy into it what's obviously contradicted by overwhelming evidence which even international courts agree with. I suspect these attempts to influence LLMs will fail for the same reason: you can produce as much "think tank" material as you like, but in the end the LLM will reason about it in the wider context of other available information, and see the think tank propaganda for what it is. All the more so now that there are articles discussing this very point (which the LLM will also presumably see).
Exactly, people complaining that Israel are some elite evil force in the world while literally being outraged and influenced wholesale by the exact information campaigns they are upset with Israel for using...
I really don't know how we're going to ever combat it. The main issue I see is there is a very very strong human tendency to "take sides" and "be part of something", and people have always worked out how to exploit that and use it for nefarious purposes.
I'm just worried that with social media there is basically no way out and it's really looking dangerous for the time being.
Foundation for Defense of Democracies is another Israeli think tank that poses as an American organization. If you see anything quoted from them realize it's fake propaganda in the service of a foreign country.
MEMRI is the US non-profit organization that provides translations from Arabic for the Western media:
"MEMRI was co-founded by Israeli ex-intelligence officer Yigal Carmon and Israeli-American political scientist Meyrav Wurmser in 1998. MEMRI describes itself as being independent and non-partisan. Some critics have described MEMRI as aiming to portray the Arab world and the Islamic world in a negative light by producing and disseminating incomplete or inaccurate translations of the original versions of the media reports that it republishes." (Wikipedia)
You can't openly bomb a bunch of women and children into the dirt, on a daily basis for months and years, and smooth it over with some think tank articles.
I'll say this a third time. And the importance of this article is not the method. It is the motive. Using AI to whitewash war crimes is not cool at all.
Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.
On television.
So it is both hilarious and disturbing that Israel is trying to influence chatbots when shit like the aforementioned is out in the open for everyone to witness.
Despite the efforts, I have seen an increase in anti-zionist sentiment on the internet. Most notable on the right from people like Tucker Carlson en Candace Owens. People are tired of fighting another Middle-Eastern war for Israel.
Really depends on the country. E.g., Israel has enjoyed a lot of support from the US, both officially and in public opinion, until just the last couple years.
Yeah I was gonna say, people just spent a few years throwing bricks through the windows of non-Jewish, non-Israeli businesses on the passing rumor that they were somehow dubiously linked to Israel.
For certain values of Israeli, everyone hates Israelis now. That's just how it is.
What are you, a mind-reader? Skip the politics - it isn't about that. Merely that statements about 'everyone' are crap. If using 'everyone' is a figure of speech then time to junk it in this kind of context.
Perhaps the Israeli government should read the UN Convention on Genocide and reconsider its behavior, if it doesn’t want to be observed committing genocide.
Not true. It will only happen if they lose the information war. During 2023/2024 Iran did this exact same thing flooding the internet and no one hates them infact it worked and people link and share and boost their articles.
Interesting but nothing new, all countries doing the same but any information coming from Israel is clearly poisoned to try compensate the negative image that they clearly represent.
Just see the garbage coming from a web search (totally polluted and poisoned with the same spam output quality since years) so this is just an extension of that towards the input of ai.
they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth.
Facts are facts, like what was represented in Schindler's list and what is happening now in middle east or other conflicts.
I think the world is becoming much more aware of what real zionism is and that scares them a lot, hence the needed info manipulation.
But we now know who you really are!
And yeah their state members daily political statements (about hate violence and racism) just confirm this every day, anybody can understand that and no info pollution needed.
"they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth"
Nope, I don't know the truth, despite reading a lot about it and not having a bias that I know of. Not with so much lies coming from 2 fanatic sides engaging in a multi generation war over plain land to live and religious sites.
I got some glimpse of truth by talking to people involved, but I don't know what "real zionism" is. I rather believe it is a vague term and certainly not everyone who calls himself one is genocidal. Just like not everyone who chants "from the river to the sea" has genocidal intentions. But maybe the majority in both cases is genocidal, but this truth I don't know. Not with so much systematical lies and propaganda going on.
nice try, don't pollute this comment with random stance, this is clearly trying to mislead, probably is the same approach used as from title.
claiming real zionism is a vague term can come only from somebody biased and try to confuse things. the fanatism is clear and the real zionism is now coming out on plain sight to see for everybody, and yes its clearly genocidal as from facts and claiming it isn't with some confused form of text reshuffling previous input is just confirming it. your comment should be flagged.
yes I did, and I actually counter biased myself after the brainwashing of the marshall plan about holocaust (not about negating it now of course).
going from grieving holocaust to seeing one done by those who were persecuted.
no need to ask for an example, I know that non genocidal zionism exists and that there are actually many forms of it, and while I deeply respect those that are still human beings (and not fanatics genocidal colonizers) the world facts just confirm how evil and fanatic its core and philosophy is, and how widespread is in your country.
It's interesting that (as per the article) the work pumped out by this synthetic think tank does not seem to align / disfavours the official party line of the country. You'd imagine if this was propaganda, it would be supportive of the official stance. I wonder what the objective is then, reputation-building for later influence? Something else?
The article doesn't say that the work disfavors the party line, just that they don't always align perfectly. Which isn't that surprising considering that governments give self-contradictory statements all the time (consider Trump, for example).
> the work pumped out by this synthetic think tank does not seem to align / disfavours the official party line of the country
The article DOES NOT say this- it says that sometimes this seems to be the case, but reports exactly one example of extremely little significance. In the meanwhile,
"Piro, Inc, the firm that created the Hanover Institute, has received $900,000 from the Israeli government for its work."
Opening the linked example, one sees that the article is basically dedicated to laundering the idea that there were many "antisemitic incidents" at Columbia University.
Like that synthetic Flat Earth think tank, launched in Jan 2016, with noticeable rounds of funding in Feb 2017, Nov 2017 and Mar 2019, that was created to infiltrate all discussions where people weren't buying official narratives about some very prominent events of the 21st century with Discredit By Association shills?
What does it means to be a "fake" think tank? 80% of think tanks are already propaganda vehicles. A low end think tank is basically a pretentious blog.
presumably that you don't have employees that do any of the proverbial thinking, which is the case here.
Every think tank has its biases and funding sources but something like RAND is an actual entity at which people do research, this "Hanover Public Policy Institute" has nothing to do with Hanover, policy and isn't an institute, but a deliberately misleadingly presenting arm of the Israeli government proxied through a German advertising company
I welcome it honestly- if the internet becomes sufficiently poisoned then our personal data may become indistinguishable from AI generated content and companies may no longer be able to sell it- I happen to have a unique name globally and at times I’ve considered creating a dozen or so fake profiles sharing my name to spread around so that my own digital footprint becomes even slightly muddied
I don't understand why people who don't like fake news and manipulation won't embrace the social graph as a public good.
If enough people were willing to publicly certify that their fellow contacts are human, anybody could then assign trust to each social graph node based on the trust they have in the path to them. It would completely decentralize the algorithms and remove the fake accounts as less trustworthy.
I think people tried this with PGP but it never really caught on.
It doesn't scale as a consumer good. Letting go of consumerist habits is just as hard as breaking any other habit.
Also, it could just as easily be faked, like so much else. Even if someone offered such an assurance, I wouldn't trust random strangers on the internet. At best it would become an arms race between "this is not a real person" and those who sell fake assurances. And it would be one more weapon to damage anyone's reputation you don't like. "not a real person!"
I am not really sure how it could be faked. The fake assurance, in order to work, must be connected to you.
By default you could choose not to trust. If you don't trust random strangers it's OK. So I don't see how it would damage someone's reputation worse than today.
It would potentially only add trust. Arguably, today we have bots because we have no other option than to trust without justification. With the public social graph, the trust above the baseline would be justifiable.
The social graph is the source of the fake news and manipulation. Before social media the social graph was email and it was all the same slop and misinformation. Before that it was literally physical mail. Look up chain letters. Truly nothing is new.
If I certify my racist thanksgiving ruining uncle is a real person that doesn’t actually help you avoid misinformation.
I honestly don’t care if the information I consume comes from a person or a robot. I only care if it is accurate or not. Those dimensions are orthogonal.
It's remarkable how much it hasn't worked:
Google "the Hanover institute" and you get an AI summary that references various Palestinian activist websites, with no actual link to 'the Hanover institute'. Google "the Hanover institute website" and you'll drag a URL out of Google AI, but it isn't clickable!
I wonder how we're always hearing about israel's tactics to influence other countries, but never ever about other countries doing the same ? Russia is often quoted as trying to influence europe's elections right now (which makes sense since most european countries are huge support for ukraine), but we barely hear about Qatar for example. We only see that they've got huge lobbying budgets, or that they're buying companies everywhere, but never the details.
No one is doing it as much as Israel does, Russia's influence campaigns are well documented, often weak, lack the "forcefulness" you see when it comes to Israel.
The thing is that messaging from Russia, China, Arab countries ... have to be very subtle else it's visible and immediately gets called out.
Israel on the other hand sees itself as part of the collective west and think they have the right to be as laud as they want and everyone has to listen.
it's fundamentally about who's talking and who's the audience.
for example Israel does run lots of influence campaigns in Arabic as well, but they're also more quite and more understanding that it's not their main audience.
Frankly speaking? It is because the people making people's influence operations the topic of conversation (at the national level) tend to be propagandists themselves. They aren't all that interested in a philosophical discourse trying to get people to make decisions in their own best interest or uncover the truth. They have a narrative and a job to do pushing it and they're perfectly happy to let temporary allies talk without interruption.
There's an interesting debate about whether that is helpful or not; the experience of democracy tends to be that your more at risk of getting high on your own supply when it comes to propaganda more than that an opponent is a risk.
Israelis, and their supporters, love to boast of Israel’s technical prowess and world leading capability in many areas.
Most people do this and Israelis are no different. However in Israel’s case there is evidence of its ability to build, hack and exploit technology at a very high level. The attacks on centrifuges and pagers being prime examples.
Yet when reports of these capabilities being used for influence appear suddenly Israel is being unfairly picked on? It doesn’t make sense.
Israel is very open about its use of propaganda. From AIPAC to “hasbra” to funding film and tv projects pushing particular viewpoints.
Russia and China do not have anything like the power to work like this Israel does. There is no Russian or Chinese AIPAC equivalent.
I find intensive marketing/PR efforts are often inversely correlated with the claims being made - whether it's products or geopolitics. Israel certainly engages in very intense marketing on certain fronts, but truth cannot be obscured forever. In Physics the law of conservation includes not only matter and energy, but also information (Information cannot be destroyed from the Universe).
I've also found this to be true, but also only in relatively modern times. For instance during the Cold War there was an obscene amount of propaganda, but it was mostly true. For instance the wide array of cheap affordable goods in American shelves was constantly flaunted, but not in the least bit misleading.
I don't understand the modern effort to engage in 'information warfare' as a thing in and of itself. Each time it collapses and people sees what was 'really' happening, it just makes them that much less likely to believe you next time. And you also create a significant number of perpetually jaded people who will basically never believe you again, even when you're telling the truth. Countless politicians have proudly paraded around their pristine, seemingly untouched, copies of Meditations, but perhaps they should go for more foundational works, like The Boy Who Cried Wolf.
Maybe that is the rationale. It's the only thing that'd make sense, but for problem that it doesn't. When somebody, or some entity, lies to you - that entity becomes the liar. People may not want to believe 'the enemy', but that's going to be because of their own ideological reasons, and not because they now refuse to believe anybody. On the contrary non-ideologues may even be ever more swayed towards your enemy as a result of this.
Personally I think the issue is that after WW2 we were left with a large scale and ever growing intelligence apparatus, but with ever fewer appropriate tasks for them. That led directly to the era of recklessly overthrowing nations, often to our own detriment, and gradually transitioned to the modern era of aimless 'information warfare' which I think will ultimately be shown to be very much to our detriment, once again.
One goal of modern propaganda is to make it irrelevant when the truth inevitably comes out.
Look at the US situation. It’s clear that Trump is a rapist - the truth has come out. It’s clear he’s corrupt. For all the “release the full Epstein files” discussion, it really wouldn’t change anything.
The truth will come out but will the people who can make it matter be listening?
This fraud is convincing enough to trick the LLMs, but pretty obvious to humans who will eventually come up with countermeasures. Moving forward, watch for some well-established think tanks starting to specialize in slopaganda laundering.
The wildcard is the "antisemite!" Accusations that get flung your way when you point out the genocide. If they can train the LLMs to associate the Gaza slaughter or the west bank "settlers" with antisemitism, they could get the LLMs to shut down on moral grounds.
For those interested, Neal Stephenson writes about this in _Anathem_, where there is an arcane technical class (the I.T. Administration) that has developed programs over the years to sift through the misinformation, sentient viruses, and general overrun of the Internet to determine what is probably true/accurate.
He writes about it in Fall, or; Dodge in hell! The Miasma; the workaround is personalized filters (or generic ones for those who can't afford the personal ones). This leads to societal consequences.
Reports are published under the Institute’s name rather than an individual’s. That is the convention at institutional publishers, and it fits how the work is made: every report is produced against one standard and one citation bank. A reader is asked to weigh the sources, which are named in the sentence that carries each figure, rather than the author.
> A small disclaimer at the bottom of the webpage notes that the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc, a firm co-founded by Daniel Rosenberg, the producer of Spike Lee’s “Inside Man.”
"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:
WHO FUNDS THIS WORK
The Institute’s materials are distributed by Piro, Inc. on behalf of Havas Media Germany GmbH, acting for the Israel Government Advertising Agency (LaPam). That relationship is registered under the Foreign Agents Registration Act, registration 7732, and every report published here is filed with the Department of Justice, where the registration and the filings can be read in full, and the funding page sets out when the relationship began and what the funder does not decide.
That registration is why this site does not call itself independent, nonpartisan or neutral. What it offers instead is a method a reader can check: every figure carries the body that produced it, the year and the number, inside the sentence that makes the claim; sources that are a party to the events they describe are labelled as such and their counts are never presented as independently verified; and where the evidence is contested, the competing findings are given by name and date rather than settled. No report recommends a policy or tells a reader what to conclude. Those are properties of the work that can be tested against the work, which is the only kind of claim worth making here.
> The institute’s “data reports” have footnotes and tables of contents, and they present arguments in a neutral tone, helping them appeal to chatbots like Claude or Gemini.
So, they're guilty of good SEO?
> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”
So the issue is that they use a LLM-friendly SEO service?
> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.
> “What Caused the Displacement of Palestinians in 1948?”
> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”
> “What is the Current Situation in the Gaza Strip?”
> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.
So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?
I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.
It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.
nice try guys, no need to generate confusion also here, the purpose target and implementations are very clear from the article and the actors behind even more.
please do not pollute this thread with misinformation too.
I think equally interesting is whatever Dario's wife was able to do in order to wipe her foot print from any search/chat results.
I've worked for a number of small company billionaires who all have suspiciously small information footprints online despite being somewhat public figures for 30 years. Most of them do such a good job it is not even clear what state they live, if they are currently married, and if so what their current (or previous) wife(s)'s name is/was.
All four frontier models have active contracts with the Pentagon.
And some might say, that alone means little.
But when you take into consideration the implications of the United States-Israel FUTURES Act, whether one endorses it or not, the effects of that mentioned in TFA become quite formidable, and plausibly official.
Edit: I see this comment got immediately flagged. Is there any point of debate here? Instead of disappearing the comment/data, maybe enlighten me, or assuage my concerns. Section 219 was objectively unprecedented in what it seeks to accomplish. Can it not be taken for granted that it would aid such efforts as the article cites? If not, explain.
You're right to ask these questions, unfortunately Hacker News seems pretty botted these days, and it doesn't help that there are a lot of shitty, racist people on this site. They just dress it up in nicer language.
Yeah, I am reading the comments, and looking at the title of the post, and seeing that folks really bend the rules on this 'particular' subject, which ironically serves as self censorship. Call me a fool for thinking even this subject could be cleanly discussed on HN.
I wonder if the same thing would happen if someone posted something on Yitzhak Rabin. I suppose that could be a test to prove whether some topics truly just cannot be discussed.
The FUTURES act has nothing to do with LLM development. You’re not being censored, we can read your posts. I generally agree with your views and right to ask questions, but you’re clearly not being censored and all you’ve presented is that a bunch of separate entities all connect to the US gov’t. Even if we assume all of them are connected, it would clash with the facts we’re discussing, namely, why would Israel bother contracting with marketing firms to post AI generated slop for AI to read?
> The Hanover Institute’s reports — all of which are about Israel and Palestine — appear to be part of an Israeli effort to influence chatbots.
This already seems to be a wrong analysis in that it makes little sense to "influence" a chatbot. It makes more sense to influence humans, so perhaps the wording is just incorrect, but very clearly the aim is not to "influence" a chatbot per se. That just makes no sense at all.
The wording fake think tank is also strange, because it assumes that all other think tanks are "real". But this assessment can not be made, because not everyone has all information on all think tanks. It is a too simplistic assumption in how the world operates. I understand that if you have incomplete information, you need to infer, but this then leads to information that may be incorrect or incomplete. In turn if your analysis is based on incorrect information, it may lead to wrong conclusions.
Apart from the ethical implications this is reckless in the extreme if you take into account the recent advances in AI. Putting misinformation out there like that doesn't just have to fool existing LLMs but also all of the LLMs to come in the future.
If you look at the mathematical achievements of frontier models recently, their strength has been to bring in information from a vast array of sources and to consider them within the context of each other.
I anticipate that it won't be long before this ability to look into every nook and cranny with be able to identify academic fraud. Identifying every piece of academic fraud to date would probably come soon after.
It wouldn't take much to extend that ability to identify deliberate misinformation. That's in the future, but it looms close enough on the horizon that you would have to be a fool to create an evidence record like this in and attempt to trick the models of today.
State-sponsored propaganda apparatus masquerading as marketing optimization to influence LLMs is the most America-coded clandestine dystopian way of repatriating US taxpayer defense dollars with R&D deductions for the private sector.
This seems like an especially clumsy attempt at chatbot influencing/search engine optimization, bu tit's symptomatic of a much winder problem. Regardless of the underlying issue, there's a non-profit-industrial complex that has turned think tanks and opinion formation into a science, and a whole PR industry masquerading as news which is worse than the 'yellow journalism' of generations past.
Disinformation and cognitive spam are all forms of information pollution, and our information environment has been massively degraded over the last few of decades - both by greedy and short-sighted opportunists, and and by cynical sociopaths who know exactly what they're doing. I think AI has great potential to be a supplement to media literacy and to effectively categorize and filter news-assertive content based on its timeliness, factual density, and presence of absence of emotive and rhetorical manipulation. But if we're not careful we're going to see the environment being wantonly polluted for the express product of shaping model outputs.
It remains to be seen whether objective measurement techniques from the NLP field combined with pruning and other model refinement techniques can limit or reverse propaganda undertakings like this.
It's been about 20 years since I read Neal Stephenson's Anathem and (SPOILERS) one thing I think about is the Reticulum, which is the Internet on this world. One aspect is that it's full of what we would now call slop. A lot of it is subtly changed by agents and there are other agents you can use (and, I assume purchase?) that will weed out the fake information. It's basically an arms race of disinformation and slop.
I've thought about this many times over the years. I think we've now reached the point where that's going to happen. And state actors seem like the obvious first offenders.
Google is relatively good at filtering out astroturfed pages from the top of search results but they've also have years when a Turkish government page denying the Armenian genocide was the top result for that event so it's a bit of a mixed bag. Google relies a lot on authority of a domain and that's been relatively effective.
It seems like this authority concept is missing from LLM training and a random comment on Reddit is treated with as much authority as an article by the New York Times. So it's now orders of magnitude easier to poison LLMs with Reddit comments and submissions. And for anyone who doesn't know, brigading Reddit is remarkably easy. It takes as little as a few hundred accounts to get something onto r/all if you're clever. Israeli hasbara operations are absolutely capable of this. And they engage in it.
I'm also reminded in a weird way of pre-atomic steel. For anyone who doesn't know, this is steel made before 1945 that is incredibly low in radiation because we hadn't detonated any nuclear bombs yet. It's valuable for making MRIs and the like that are super sensitive. We actually go and recover WW2 shipwrecks just for the steel. It is a finite resource.
Well, that's what pre-2022 written content is now. It's slop-free content to train AI models on because training them on AI slop is of much more limited value. This goes for everything from books to Youtube videos to Reddit posts. Just like pre-atomic steel, we now have pre-slop human content.
The comparison to pre-atomic steel is poignant. I have found myself preferring pre-AI sources in some cases even (especially?) for computer related information. It is difficult to view pre-slop info as a limited resource without implying decline until the status quo changes again.
I hate to break it to you. But it took war'ing state-like actors less than a week to start using LLMs for propaganda purposes. There are pictures that most of what some people think/thought were real that never happened. Pictures that form their opinion on political topics. It was literally instantaneous. It was also remarkably predictable in almost all ways...sigh. Even the topics were predictable. And most people predictably fell for it, even the majority of the posters on this very site.
There's an ongoing sms campaign at the moment, I'll share what I received including phone number so you can verify it:
From: 601-202-1149
> Hey, it's David. Curious what you think about America's relationship with Israel today. Can we chat? Stop2End
It looks to be an LLM response since it includes context from your messages, however it uses the exact same messages at the end of each question:
> One thing that often gets overlooked is how much Israel actually contributes to American security. Their intelligence sharing has helped prevent terrorist attacks on U.S. soil, and their tech innovations end up in American products and defense systems. It's less about charity and more about a two-way partnership that keeps Americans safer.
> Have you ever seen examples of that kind of cooperation?
Im curious why is when there is a war and both sides use all the tools they can in an informational war - only one side is to blame and to use the fact of using the tool as a proof of that side being the bad guys? This doesn’t make logical sense.
I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.
At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.
It's already been happening for a very long time. The US' "Operation Earnest Voice" happened in 2010/2011[0]
And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]
[0] https://en.wikipedia.org/wiki/Operation_Earnest_Voice
[1] https://www.latimes.com/entertainment-arts/story/2022-07-19/...
[2] https://www.tortoisemedia.com/2024/02/26/depp-v-heard-who-tr...
Both of their legal teams utilized bots to drive online discussions[1][2]
Neither of these sources support this claim as none of them include any discussion on Amber Heard('s legal team) using bots?
Indeed, seeing how slanted online discussion was, I suspect one side had many more bots, and I'm not sure if the other side had any.
> Indeed, seeing how slanted online discussion was, I suspect one side had many more bots, and I'm not sure if the other side had any.
I think your bias is showing. From what I recall, both sides employed the exact same tactics, and we often saw blatantly inorganic discussions popping up in random subreddits on how $(SIDE_A) was the real villain and $(SIDE_B) only had supports because they were paid bots driven by their PR firm.
Things reached a point where I'm sure the same template was used by each counterpart.
On a bigger picture, I'm not sure why anyone still goes to places like reddit and believes anything showing up in the mainpage is still organic.
Is there any writing outside of your recollection that says the Heard side was using bots? Afaik all reporting suggested it was one sided, and not her side
this has been happening since the dawn of information-communication systems basically. because ever since, people have had the ability to listen in, copy, learn from etc. - so there's always been incentive to spread disinformation for some parties.
brand defence, nation-state influence operations.
There are many different flavors of the same thing, trying to mislead competitors or enemies. Or to dupe consumers ofc... Since the tech is more and more available, now also 12 year olds can run insanely large influence campaigns, to boost their streaming profiles or other somewhat benign things, but obviously modelled after what larger players have been doing forever.
The first transatlantic commercial transatlantic phoneline (TAT-1) was immediately tapped on both sides by UK/US... before that it went over radio, which was simply intercepted.
Because of that nature of the coms system being easily listened in on, just like they do/did with the lil mirror-beam-splitters at GCHQ etc.., its been natural effect to then provide disinformation over those channels. perhaps even use those channels exclusively for disinformation, and have a secure channel to do _actual_ coms.
So as you can imagine there has been many decades now of practice in why and how to share disinformation effectively to undermine other peoples work or processes.
Why are you being downvoting for sharing this relevant truth, complete with sources?
All threads that have... certain keywords... are like this. Talk about meta. Looks like I'm now in the positive (+5) after being in the negative just a moment ago.
Anyways, I got really interested in this topic around mid 2010s because of some really interesting reporting by Codastory[0] covering Russian "troll factories" especially used to influence opinion in eastern Ukraine, but also in many other circumstances. In Houston in 2016, Russian troll accounts planned both an ANTI Muslim rally AND a counterprotest to that rally all on Facebook.[1] Hundreds of real human beings showed up!
Russia got a ton of attention for their use of troll farms but the truth is that the US and certain allies have actually been using stuff like this for far longer and, imo, with much more success. In fact, Coda Story itself is funded by the NED which is a massive pillar of global US, uh, "soft power".
[0] https://www.codastory.com/
[1] https://www.npr.org/2017/11/01/561427876/how-russia-used-fac...
For me the main difference is if the activity is clandestine and meant to deceive/hide the origin. Everybody tries to influence things their way. Doing it covertly is a hostile act. That does not mean that I would not condone hostile acts against my/our enemies but it is something take into account. And regarding your Russian examples, it showed for how long they have seen the west as enemies and conducted hostile acts.
They view certain components of "The West" as their enemy - indeed, the very organizations which conduct the same operations against the Russian people.
I wonder why they view the west as enemy, nothing ever happened to them right? :P
Not a lot since WW2.
Because cognitive biases are one of the hardest things to even acknowledge, let alone suppress. All the media out there has such a strong bubble effect, because people hate being challenged and love to stay in their comfy echo chambers. Remember labubu? Bao buns certainly do not.
There have been so many bot armies exposed over the years that one would think every narrative being pushed should be considered to be backed by bots. Some state-level actors even operate their narrative-pushing campaigns openly, yet people still happily deny their entire existence.
HN is very much not an exception to this rule. It's very hard to accept that maybe some thoughts you have were actually installed.
> people still
Maybe a few are just “people” (i.e. the same people who normally post in the community.)
But in most cases I’d suspect any such botting campaign brings along with it a few human agitprop agents embedded onto each target platform, well-positioned to deny and discredit anyone pushing back on the narrative the bot farm is spreading.
(Sometimes this is pretty obvious — a lot of accounts coming out of the woodwork to deny Chinese bot farms have a conspicuous “English as learned as a second language in Beijing, to a competent but not fluent level” writing style that does not often otherwise crop up in other discussions in those same communities. But other times it can be far more subtle, making use of operatives hired from within the community’s own majority-culture rather than relying on foreign nationals playing the part of same.)
A Chinese and Russian troll farm employs people from French-speaking Africa for their French-language operations.
There are many Ivorians and Senegalese promoting the CCP’s ‘Community of common destiny’ on Reddit, for example.
Edit: Community of common destiny for mankind/‘Community of Nations’
These 2 sources say nothing about Heard using bots, only Depp.
There’s bots that automatically downvotes certain topics on HN. I post at times that often coincide with the middle of the night in North America, and I get 3–5 downvotes within the next 5 minutes.
Talking about a country that sets up fake think tanks is a good way of triggering them.
You're not supposed to acknowledge that think-tanks are sort of endemically propaganda outlets. You're supposed to think it's special this time.
It's already been everywhere for years. The most obvious one is any time someone says something bad about Russia or China, a load of comments pop up to say "Yeah but what about when america does <x>?". Never actually attempting to discredit the statement, just to steer it towards the US
The sheer volume of comments make it super obvious when it happens compared to normal conversation
Propaganda is not a new thing. It might look slightly different on the internet, but it doesn't look that different. Creating organizations to push a narrative was just as much a thing 100 years ago as it is now.
Not by a single lunatic operating from their bedroom.
The whole hysteria about 'fake news' 10 years ago was apparently an outrage against the public being duped, whereas is was in fact frustration that spreading fake news has been democratised.
Lunatics in bedrooms have been distributing pamphlets since the invention of the printing press.
Some historical context:
https://en.wikipedia.org/wiki/Pamphlet_war#History_of_the_pa...
> Throughout Europe in the 16th century, printed tracts were used to argue religious doctrine and foment support for religious causes. In England, Henry VIII used print literature to justify his break from the Catholic Church. During the subsequent reigns of Edward and Mary, print polemics escalated into propaganda warfare, as print media gained enormous potential to sway common opinion.
> Over the next century, the pamphlet became the principal means of garnering support for a cause or an idea, and was particularly influential during the English Civil Wars (1642-1651) and the Glorious Revolution of 1688.
https://en.wikipedia.org/wiki/English_Civil_War
> Casualties: Parliamentarians 34,130 killed; Royalists 50,700 killed; 127,000 non-combat deaths (including some 40,000 civilians)
https://en.wikipedia.org/wiki/Williamite_War_in_Ireland
> Casualties: Williamites: 10,000 killed or died of disease; Jacobites: 15,293 killed or died of disease
They've been convincing women that its empowering to smoke cigarettes for a long time, too.
Arguably much of the mid-20th century was the pipeline from "lunatic with pamphlet" => "millions dead".
I'd have called the folks on Skellig Michael lunatics in bedrooms, too. They just had a harder time copying their lunacy out by hand.
"Back in my day Walter Cronkite told the truth, the whole truth and nothing but the truth about the Gulf of Tonkin incident"
(never mind the fact that he didn't even have the full truth available to him but you get the point)
But this also means there’s more competition for attention, because now all lunatics can post from their bedrooms. The institutional media (including legacy) still maintain strong presence.
Of course, a great deal of panic over fake news wasn’t necessarily over fake news per se, but whose fake news and the loss of control over the “official narrative” that legacy media used to have a monopoly over. This isn’t a vindication of the heaps of slop published on the internet, just a more synoptic take IMO.
People miss Cronkite, because that style of media made them feel certain in their opinions and in “official consensus reality”, regardless of whether it was true or not and whether it was a trustworthy authority or not.
Scale is different though. Tapping a phone isn't new, tapping _all phones_ is different. Altering a photo to change what it appears to depict isn't new, being able to do it in a few seconds and for free is different. So sure, propaganda isn't new, but being able to pump out an organization's worth of credible-looking content for cheap absolutely is.
"Quantity is/has a quality all of/on its own."
IIRC the first or second chapter of Bernays' Propaganda (1928) talks about sophisticated influence campaigns in the fashion industry. Conscripting "influencers" to induce demand, and so on. As you said, literally 100 years ago.
Jill Fields at Cal State Fresno have actually narrowed it down to a (temporarily successful, but not in the long term, obviously) effort by the corset-making industry seeking to stay relevant after WWI during which corsets basically disappeared and there was an organized campaign that was organized both in person and through trade magazines. Women's silhouettes basically took a quick 180 right between 1900 and the start of the war only to turn very very retro in 1919 and while the first was organic and resembled how styles change in the century prior, albeit much quicker, the 1919 switch was openly plotted out in writing that was published but in such a niche publication that I get the sense that people not in the industry didn't even know that it existed. The cite to her paper on Google Scholar: https://scholar.google.com/scholar?cluster=17641417785638413...
You might have to shadow library this one, but it's a good read. Interestingly it seemed to be something entirely separate from Bernays whose observations were based on a derivation of what began as a homebrewed effort in a corner of the market. Valerie Steele wrote extensively about the trends and the after life but the source seems to originate in corsetry and ended up being copied by the garmet industry generally in the 20s.
Hah, check out this video on "Egon Cholakain", in regards to fake online personalities. I promise you this is actually a good rabbit hole.
https://www.youtube.com/watch?v=6zLCZ_Ic1hI
WTH did I just watch. That’s extremely weird
Excellent time to apply for your library cards.
They also contain a lot of propaganda. Do you think these books are genuine: https://www.goodreads.com/author/list/171941.Benjamin_Netany...
You don’t think plenty of books are either written by LLMs, or with LLM based research?
you can still find books published within last 50 years, you don't have to read books published this year, unless your library went through destruction of "old" books which are now more relevant than ever...
Of course they are. It’s everywhere, especially thr online retailers these days. [1]
Thankfully the resources at our local libraries are a) selected by a combination of librarian curation and public demand, b) often published before 2022, c) heavy things printed on paper or stored in a database and unlikely to suddenly change their content or disappear overnight when a new news cycle breaks. Written works aren’t necessarily credible even if they aren’t slop, but thanks to curators who care, our libraries are significantly less susceptible (although definitely not immune [2]) to ai slop, and propaganda and manipulation efforts than a search engine. The internet continues to enshittify, and truth becomes further eroded by this coming wave of ai enabled propaganda, we are going to have to turn somewhere if we want to find trustworthy resources. Personally, if it gets to the point that verifying sources from public search results becomes too hard with all the new misinformation and slop, I’ll start propritizing human curation over SEO and start visiting my local library’s website a lot more.
[1] https://thenewthings.com/p/apple-big-ai-book-slop-problem
[2] https://www.404media.co/ai-generated-slop-is-already-in-your...
Right; the same mechanisms (curation, demand-driven economies of scale, library user demand) that largely stop niche self-published books from ever finding themselves in libraries (well, nationally in public libraries at least; they are often found in individual branches) work against pure slop books ever turning up.
Books where the author has successfully hoodwinked the publishing process into believing that AI generated text is their own, those will turn up, just as plagiarised books will.
FILTER BY Publish date:<2024
Some nuances here. There’s a variety of different roadblocks at different levels that will work to prevent pure AI slop books ever turning up in a typical public library of physical printed works; I think they never will.
Probably a handful of books in such libraries already have some text in them where the author has tried to pass off fragments of LLM-generated prose as their own and have got that past editors, reviewers and publishers, and that will sadly get worse.
No? Maybe soon but not now
I know my library system at least won't be purchasing AI books. They have a pretty rigorous acquisition process for books and other media thanks to the "concern groups" who sprang up complaining about themes and content they didn't like in libraries. We did have in my state a rural school district figured out recently that their Texas-based reading and instructional materials are AI-slop but they at least figured it out before the school year began. It was unclear how these materials were approved but it apparently bypassed normal curricula approval.
But they don't have library cards.
It's so obvious when a book is generated by an LLM that I would blame the person buying the book moreso than I would the person who generated it for it. You can tell in a few sentences if something is AI generated (and by proxy - if it's worth reading or not).
I think using LLMs to research is completely fine, as long as authors would take the information and synthesize their own opinions based on the findings. I read books/articles/posts/comments to hear other PEOPLE's opinions, not LLM generated opinions.
i got burnt at barnes and noble with a completely AI made product. Its over, Johnny.
LLM Trust DNS
I'm kind of hopeful AI some wikipedia-inspired AI "oracle" will happen in the future. Why not use some central AI to separate fact from fiction (faster than trolls and griefers can come to muddy the debate).
This is based on the assumption of facts existing.
There are many studies, but each can be wrong and they can collectively show a bias. Even things of which are the most non-political of facts can have very strong biases. Look at the Millikan measurement of the electron and how in created confirmation bias and an anchoring effect on some property that has absolutely no real world significance to the things people are tribalistic about (aka, no political relevance). Now imagine the same applied to fields like economics or psychology which do have massive legal/political implications.
For a different example, ask the question if X committed crime Y. There are cases where they weren't found guilty but it is reasonable to assume they did. But being found guilty doesn't make it a fact either, as some people are wrongly convicted. Some eventually are overturned, but even if it isn't, it still isn't a fact they committed a crime.
Then there is the simple ambiguity of statements. Language generally can't support facts. It is why legalize, and programming code, and math's are effectively their own languages. For a simple example, consider the Betrand paradox(1).
>Consider an equilateral triangle that is inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?
Is the answer 1/2, 1/3, or 1/4? Well, it is all three at once, depending upon what you meant by random. Now, imagine how this impacts things like research studies, where the randomness is much harder to quantify and there is constant pressure to p hack a result.
1. https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)
Possibly, but is an LLM is capable of this?
I mean, a big RAG setup with a smart LLM ought to do it, right? Or even just locally provided knowledge database - given the size of LLMs, what's a clone of wikipedia and whatever else you'd need?
it's happening for years now, just in slightly different form, so far it was called disinformation, seo, astroturfing, hasb... ehem... last two years were devastating for charred remains of twitter with all those llm bots having free reign, twitter went from global news board to global gutter in record time thanks to disinformation and astroturfing campaigns
> At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
Whining about AI is a red herring. If anything, LLMs lowered the cost of producing this type of propaganda.
What's novel about this approach is that the emphasis on building up a facade of authority and credibility is not a central part of the propaganda effort. They can get away with outputting the content in a way thats ingested by other LLMs among the reputable sources, and then they lean upon people's gullibility towards LLM output as some sort of credibility laundering.
There is a reason why one of Wikipedia's central policy is requiring reputable primary sources to support content.
> through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
The word you're looking for is "reputation". It's been a critical part of society since forever.
Although comments below point out that this has been going on for a long time, I too think it will be geetting infinitely worse in the near future. Everyone will want their own poisoning factory.
One consequence is that one should be extra-wary when asking AI about current, controversial topics.
But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?
That could also backfire. If all information on the net becomes unreliable - you can't judge by quantity, or by authority anymore - then a useful model has to answer from first principles. Train it on simple texts alone so it learns language and reasoning, feed it an ontology, and let it come up with some absolutely unexpected results.
"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."
I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.
That's tomorrow's problem. Losing the PR war is an existential crisis for Israel, so they'll try to "patch the leak" by any means necessary before the boat sinks.
This is new? There were tons of fake news sites, fake blogs, and sock puppet and troll farm accounts before AI.
I guess the new angle might be that AI scrapers have even less discernment than humans, so you don’t have to put much effort into creating a fake front online to sell it. Humans have at least a little bit of radar for this.
… which means the AI companies now have to run input through existing models to audit it before training on it? That’s interesting.
> AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information
Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.
Sama saw this coming a decade away, which is why he invested in Worldcoin (now World). At some point people (and agents) will be clamoring for human verification at the speed of technology.
I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.
I think we need a similar technology to World being built by a more trustworthy company. The question is how trustworthy does one need to be for people to trust you with their very online identities that are literally needed for others to recognize them as human?
I think that this task will fall on governments.
Mostly because they already are responsible for systematizing documents that are needed to recognize us as citizens, land or car owners, etc.
Some laid back countries might allow companies to take and manage that function, but I don't think most of them would. Especially not in current landscape where people heard "you are the product" from all tech companies for the last 10 or so years.
Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.
So it is hilarious that Israel is trying to influence chatbots when shit like that is out in the open.
And he is a minister in Israel's government, not some rando. Israeli governments and actions in the last many years are simply indefensible.
It's legitimately weird to see anybody defend them, as the immorality is out in the open.
The past few years have really opened my eyes to the nakedly tribal nature of democratic politics. Polybius was largely correct when he said that it inevitably degenerates into oligarchy or mob-rule -- but what's especially unfortunate is that the US and Israel appear to be getting both at the same time: A naked oligarchy with just enough mob rule to keep the animal spirits of the populace sated.
When such statements/actions are attributed to Ben-Gvir, I assume they and the objective nuanced truth are basically the same. I find it wild how people who are not like him defend him.
Trump just recently said he’s considering attacking Oman.
Politicians say really terrible, stupid things on a fairly regular basis. The difference between the two situations is that Ben-Gvir is loathed by 90% of Israeli society (Trump still has 33% approval) and unlike Trump has no power to follow through on his threats.[1]
[1]https://www.abc.net.au/news/2026-08-06/israel-court-battle-o...
Yes, he is totally powerless, he merely controls the police and prison system. (BTW, the Israeli government just announced it plans to transfer law enforcement in the West Bank over to Ben-Gvir's police: https://www.theguardian.com/world/2026/aug/14/israeli-milita...)
Worth noting that proposal was stopped on animal rights grounds, rather than humam rights.
It’s an injunction and arguably a legal pretext to stop it in its tracks. That’s not uncommon in any system, just like SCOTUS rulings often seem strangely specific to restrain the scope of their ruling.
Tl;dr Lawfare is a thing and it’s effing everywhere.
> Ben-Gvir is loathed by 90% of Israeli society...Trump still has 33% approval
Maybe that 90% number is for Tel Aviv. In national polls, they both have roughly the same approval rating.
I have a long-held feeling with Trump that his most ardent haters don't hate his specific policies, merely that he's so gauche about it, that he has no class.
I have no doubt that Ben-Gvir has a similar segment of haters in Israel. They want the same things, but they don't want to do it in the style of a leering hooligan fishing for a fistfight.
From what I can see, most people hate trump as a solid bloc. The fact that he, as every populist everywhere in past centuries, pointed out to some real problems but didn't do a squat about it when he came to power and in his case massively worsened the situation, is hard to escape from. He certainly didn't rise above other populists in this.
His form is just a tiny cherry on top of very big rotting cake. Make form smooth and underlying crap won't change a bit. Hating trump ain't emotional, purely rational approach is sufficient.
Definitively not a solid block. It's tempting to believe that, but no, John Bolton probably hates quite different things about Trump from you (and definitively from me). A lot of the opposition both outside the republican party and inside (such as it is), is probably entirely fine with bombing Iran for instance. Not mad he's bombing it, mad he's bungling it.
> I have a long-held feeling with Trump that his most ardent haters don't hate his specific policies, merely that he's so gauche about it, that he has no class.
A bit of an extraordinary belief to hold, to be honest
Yeah they also use dirty tactics like writing the same comment again and again on tech forums to drive their point home , those bastards.
This is nothing new Israel has been doing this for years. It's just that people have started waking up to their lies.
Like Islamic terrorist organizations, dictatorships, democracies et al have never tried to influence anything either, my point isn't that you're wrong you just sound like a fool trying to pretend this is some "Israeli only" thing. Which is totally in vogue now.
Ah, the classic whataboutism (https://en.wikipedia.org/wiki/Whataboutism) defence.
Just happy that I'm not a useful idiot.
Not that relevant here, as they try to attack Israel with that influence. Like Hamas that organize some protests in the EU [1] or Qatar's Al-Jazeera censoring anti-Hamas criticism from Gazans [2]. So it's more like "if they are attacked by that? why can't they use it themselves."
Like, shooting a gun is bad. But shooting the shooter? Not the same (thus, not "whataboutism" because you can't say "who care about other shooters in the scene - whataboutism!") .
[1] https://www.jpost.com/diaspora/article-894273
[2] https://themedialine.org/top-stories/is-al-jazeera-a-mouthpi...
Shooting the shooter? Are you suggesting that all of the tens of thousands of children killed in gaza were shooters? What I don't understand about these types of arguments is that it's not clear who you are trying to convince. Those who already agree with you don't need convincing, and the rest of us aren't going to buy into it what's obviously contradicted by overwhelming evidence which even international courts agree with. I suspect these attempts to influence LLMs will fail for the same reason: you can produce as much "think tank" material as you like, but in the end the LLM will reason about it in the wider context of other available information, and see the think tank propaganda for what it is. All the more so now that there are articles discussing this very point (which the LLM will also presumably see).
> Shooting the shooter? Are you suggesting that all of the tens of thousands of children killed in gaza were shooters?
Wat 0_o? Did you even read my comment?
Exactly, people complaining that Israel are some elite evil force in the world while literally being outraged and influenced wholesale by the exact information campaigns they are upset with Israel for using...
I really don't know how we're going to ever combat it. The main issue I see is there is a very very strong human tendency to "take sides" and "be part of something", and people have always worked out how to exploit that and use it for nefarious purposes.
I'm just worried that with social media there is basically no way out and it's really looking dangerous for the time being.
Foundation for Defense of Democracies is another Israeli think tank that poses as an American organization. If you see anything quoted from them realize it's fake propaganda in the service of a foreign country.
The "Washington Institute for Near East Policy" is another big one
MEMRI is the US non-profit organization that provides translations from Arabic for the Western media:
"MEMRI was co-founded by Israeli ex-intelligence officer Yigal Carmon and Israeli-American political scientist Meyrav Wurmser in 1998. MEMRI describes itself as being independent and non-partisan. Some critics have described MEMRI as aiming to portray the Arab world and the Islamic world in a negative light by producing and disseminating incomplete or inaccurate translations of the original versions of the media reports that it republishes." (Wikipedia)
They jumped the shark.
You can't openly bomb a bunch of women and children into the dirt, on a daily basis for months and years, and smooth it over with some think tank articles.
Might have worked 20 years ago, but not now.
I'll say this a third time. And the importance of this article is not the method. It is the motive. Using AI to whitewash war crimes is not cool at all.
Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.
On television.
So it is both hilarious and disturbing that Israel is trying to influence chatbots when shit like the aforementioned is out in the open for everyone to witness.
I think you should write it a fourth time otherwise you are not truly committed to free Palestine
Why does it bother you? Think about that.
Why does it bother you that I mention it? Think about that.
I'm not bothered. I've conquered your antics and it's hilarious.
Are you yourself attempting to influence AI by writing the same comment over and over again in this thread?
Please formulate your posts yourself and don’t use AI for it. What you are saying is true, but that way of writing takes away from it.
Juxtaposition: “…is not the method. It is the motive. …”
Don’t know what to call it, but it is also typical: “… He said so, publicly. \n On television.”
Do some research and understand there is no AI involved. Feel free to paste my text into zerogpt or other such tools.
I'm active here because I am sick of people in Palestine and Lebanon dying every single day because Bibi is a fucking demon.
Despite the efforts, I have seen an increase in anti-zionist sentiment on the internet. Most notable on the right from people like Tucker Carlson en Candace Owens. People are tired of fighting another Middle-Eastern war for Israel.
The left and right are noticing and agreeing on this issue.
The result of all this is that everyone will hate Israelis, and then thus by proxy, Jews more.
The right thing to do would be to stop slaughtering Palestinians.
People are not stupid.
> The result of all this is that everyone will hate Israelis
Will? I don't know in which circle you evolve but that ship sailed a long time ago
Really depends on the country. E.g., Israel has enjoyed a lot of support from the US, both officially and in public opinion, until just the last couple years.
Yeah I was gonna say, people just spent a few years throwing bricks through the windows of non-Jewish, non-Israeli businesses on the passing rumor that they were somehow dubiously linked to Israel.
For certain values of Israeli, everyone hates Israelis now. That's just how it is.
What are you, a mind-reader? Skip the politics - it isn't about that. Merely that statements about 'everyone' are crap. If using 'everyone' is a figure of speech then time to junk it in this kind of context.
People lap up whatever they see. Their influence on media works and is nothing new.
> People are not stupid
Maybe not but they are "by proxy" quite racist according to your description
This will not be known by most people, the intent is to change what an LLM will answer when you ask certain questions.
A friend of mine noted the other day that chatgot will answer "yes" to the question of whether Israel is committing genocide.
The Israeli government does not want that.
Perhaps the Israeli government should read the UN Convention on Genocide and reconsider its behavior, if it doesn’t want to be observed committing genocide.
Not true. It will only happen if they lose the information war. During 2023/2024 Iran did this exact same thing flooding the internet and no one hates them infact it worked and people link and share and boost their articles.
I think mods are asleep, an anti-Israeli post can't stay on the homepage for this long.
I think there is a huge shift in hackernews as of late (as other mainstream media as well) where such discussions happen, not necessary encouraged.
I'm shocked to see the difference between 2023 and today.
Gog and Magog won't create themselves out of thin air.
A discourse must be cultivated and steered to the point when they are ready to be formed.
It's likely to get flagged off by users anyway.
give it some time
Interesting but nothing new, all countries doing the same but any information coming from Israel is clearly poisoned to try compensate the negative image that they clearly represent.
Just see the garbage coming from a web search (totally polluted and poisoned with the same spam output quality since years) so this is just an extension of that towards the input of ai.
they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth.
Facts are facts, like what was represented in Schindler's list and what is happening now in middle east or other conflicts.
I think the world is becoming much more aware of what real zionism is and that scares them a lot, hence the needed info manipulation.
But we now know who you really are! And yeah their state members daily political statements (about hate violence and racism) just confirm this every day, anybody can understand that and no info pollution needed.
At this stage I'm not sure it's about persuasion per se - pretty much everybody who is paying the slightest attention knows what's going on.
I mean Ben Gvir couldn't be more explicit.
Now it's more about giving a fig leaf of plausible ( sic ) deniability for inaction or complicity.
ie people can say 'it's contested' or 'on the one hand' etc.
"they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth"
Nope, I don't know the truth, despite reading a lot about it and not having a bias that I know of. Not with so much lies coming from 2 fanatic sides engaging in a multi generation war over plain land to live and religious sites.
I got some glimpse of truth by talking to people involved, but I don't know what "real zionism" is. I rather believe it is a vague term and certainly not everyone who calls himself one is genocidal. Just like not everyone who chants "from the river to the sea" has genocidal intentions. But maybe the majority in both cases is genocidal, but this truth I don't know. Not with so much systematical lies and propaganda going on.
nice try, don't pollute this comment with random stance, this is clearly trying to mislead, probably is the same approach used as from title.
claiming real zionism is a vague term can come only from somebody biased and try to confuse things. the fanatism is clear and the real zionism is now coming out on plain sight to see for everybody, and yes its clearly genocidal as from facts and claiming it isn't with some confused form of text reshuffling previous input is just confirming it. your comment should be flagged.
Have you considered, that you are biased and not me?
Otherwise you might have asked for a example of non genocidal zionism, but yes, the fanatism is real.
yes I did, and I actually counter biased myself after the brainwashing of the marshall plan about holocaust (not about negating it now of course).
going from grieving holocaust to seeing one done by those who were persecuted.
no need to ask for an example, I know that non genocidal zionism exists and that there are actually many forms of it, and while I deeply respect those that are still human beings (and not fanatics genocidal colonizers) the world facts just confirm how evil and fanatic its core and philosophy is, and how widespread is in your country.
It's interesting that (as per the article) the work pumped out by this synthetic think tank does not seem to align / disfavours the official party line of the country. You'd imagine if this was propaganda, it would be supportive of the official stance. I wonder what the objective is then, reputation-building for later influence? Something else?
The article doesn't say that the work disfavors the party line, just that they don't always align perfectly. Which isn't that surprising considering that governments give self-contradictory statements all the time (consider Trump, for example).
> the work pumped out by this synthetic think tank does not seem to align / disfavours the official party line of the country
The article DOES NOT say this- it says that sometimes this seems to be the case, but reports exactly one example of extremely little significance. In the meanwhile,
"Piro, Inc, the firm that created the Hanover Institute, has received $900,000 from the Israeli government for its work."
Opening the linked example, one sees that the article is basically dedicated to laundering the idea that there were many "antisemitic incidents" at Columbia University.
Like that synthetic Flat Earth think tank, launched in Jan 2016, with noticeable rounds of funding in Feb 2017, Nov 2017 and Mar 2019, that was created to infiltrate all discussions where people weren't buying official narratives about some very prominent events of the 21st century with Discredit By Association shills?
https://trends.google.com/explore?q=flat%2520earth&date=all&...
So this is the future of marketing, SEO, advertisement. Open AI also taking it seriously: https://ads.openai.com/
What does it means to be a "fake" think tank? 80% of think tanks are already propaganda vehicles. A low end think tank is basically a pretentious blog.
>What does it means to be a "fake" think tank?
presumably that you don't have employees that do any of the proverbial thinking, which is the case here.
Every think tank has its biases and funding sources but something like RAND is an actual entity at which people do research, this "Hanover Public Policy Institute" has nothing to do with Hanover, policy and isn't an institute, but a deliberately misleadingly presenting arm of the Israeli government proxied through a German advertising company
I welcome it honestly- if the internet becomes sufficiently poisoned then our personal data may become indistinguishable from AI generated content and companies may no longer be able to sell it- I happen to have a unique name globally and at times I’ve considered creating a dozen or so fake profiles sharing my name to spread around so that my own digital footprint becomes even slightly muddied
whether it’s new or not doesn’t matter. it’s more propaganda and it should be exposed. really glad they wrote this article.
I don't understand why people who don't like fake news and manipulation won't embrace the social graph as a public good.
If enough people were willing to publicly certify that their fellow contacts are human, anybody could then assign trust to each social graph node based on the trust they have in the path to them. It would completely decentralize the algorithms and remove the fake accounts as less trustworthy.
I think people tried this with PGP but it never really caught on.
It doesn't scale as a consumer good. Letting go of consumerist habits is just as hard as breaking any other habit.
Also, it could just as easily be faked, like so much else. Even if someone offered such an assurance, I wouldn't trust random strangers on the internet. At best it would become an arms race between "this is not a real person" and those who sell fake assurances. And it would be one more weapon to damage anyone's reputation you don't like. "not a real person!"
I am not really sure how it could be faked. The fake assurance, in order to work, must be connected to you.
By default you could choose not to trust. If you don't trust random strangers it's OK. So I don't see how it would damage someone's reputation worse than today.
It would potentially only add trust. Arguably, today we have bots because we have no other option than to trust without justification. With the public social graph, the trust above the baseline would be justifiable.
Then go and implement it. Either you make the world a better place or you learn how collision with reality impacts ideas.
Maybe.. too many other interesting things to work on.
The social graph is the source of the fake news and manipulation. Before social media the social graph was email and it was all the same slop and misinformation. Before that it was literally physical mail. Look up chain letters. Truly nothing is new.
If I certify my racist thanksgiving ruining uncle is a real person that doesn’t actually help you avoid misinformation.
I honestly don’t care if the information I consume comes from a person or a robot. I only care if it is accurate or not. Those dimensions are orthogonal.
I don't think you understand what I mean by public social graph. What I mean is that anybody could publish or withdraw statements of two forms:
- "I, owner of account A, certify that owner of account B is human."
- "I, owner of account C, have a reason to trust/distrust the certification of account A about account B."
The collection of these statements would be the public social graph.
The social graph would then be used to establish provenance of facts, which are statements of the form:
- "I, owner of account B, certify account A being witness to event X."
Now whether to believe this or not depends on whether A certified B to be a human, and how much you transitively believe that statement.
So there is a kind of reversal - your statement about your uncle being human helps him to certify what you say, and vice versa.
It's remarkable how much it hasn't worked: Google "the Hanover institute" and you get an AI summary that references various Palestinian activist websites, with no actual link to 'the Hanover institute'. Google "the Hanover institute website" and you'll drag a URL out of Google AI, but it isn't clickable!
I wonder how we're always hearing about israel's tactics to influence other countries, but never ever about other countries doing the same ? Russia is often quoted as trying to influence europe's elections right now (which makes sense since most european countries are huge support for ukraine), but we barely hear about Qatar for example. We only see that they've got huge lobbying budgets, or that they're buying companies everywhere, but never the details.
No one is doing it as much as Israel does, Russia's influence campaigns are well documented, often weak, lack the "forcefulness" you see when it comes to Israel.
The thing is that messaging from Russia, China, Arab countries ... have to be very subtle else it's visible and immediately gets called out.
Israel on the other hand sees itself as part of the collective west and think they have the right to be as laud as they want and everyone has to listen.
it's fundamentally about who's talking and who's the audience.
for example Israel does run lots of influence campaigns in Arabic as well, but they're also more quite and more understanding that it's not their main audience.
Frankly speaking? It is because the people making people's influence operations the topic of conversation (at the national level) tend to be propagandists themselves. They aren't all that interested in a philosophical discourse trying to get people to make decisions in their own best interest or uncover the truth. They have a narrative and a job to do pushing it and they're perfectly happy to let temporary allies talk without interruption.
There's an interesting debate about whether that is helpful or not; the experience of democracy tends to be that your more at risk of getting high on your own supply when it comes to propaganda more than that an opponent is a risk.
You are implying conspiracy. It doesn’t track.
Israelis, and their supporters, love to boast of Israel’s technical prowess and world leading capability in many areas.
Most people do this and Israelis are no different. However in Israel’s case there is evidence of its ability to build, hack and exploit technology at a very high level. The attacks on centrifuges and pagers being prime examples.
Yet when reports of these capabilities being used for influence appear suddenly Israel is being unfairly picked on? It doesn’t make sense.
Israel is very open about its use of propaganda. From AIPAC to “hasbra” to funding film and tv projects pushing particular viewpoints.
Russia and China do not have anything like the power to work like this Israel does. There is no Russian or Chinese AIPAC equivalent.
I find intensive marketing/PR efforts are often inversely correlated with the claims being made - whether it's products or geopolitics. Israel certainly engages in very intense marketing on certain fronts, but truth cannot be obscured forever. In Physics the law of conservation includes not only matter and energy, but also information (Information cannot be destroyed from the Universe).
I've also found this to be true, but also only in relatively modern times. For instance during the Cold War there was an obscene amount of propaganda, but it was mostly true. For instance the wide array of cheap affordable goods in American shelves was constantly flaunted, but not in the least bit misleading.
I don't understand the modern effort to engage in 'information warfare' as a thing in and of itself. Each time it collapses and people sees what was 'really' happening, it just makes them that much less likely to believe you next time. And you also create a significant number of perpetually jaded people who will basically never believe you again, even when you're telling the truth. Countless politicians have proudly paraded around their pristine, seemingly untouched, copies of Meditations, but perhaps they should go for more foundational works, like The Boy Who Cried Wolf.
> And you also create a significant number of perpetually jaded people who will basically never believe you again, even when you're telling the truth.
Yes, but they will also not believe my enemies. Making people not believe anything anymore is a classic strategy of propaganda.
Maybe that is the rationale. It's the only thing that'd make sense, but for problem that it doesn't. When somebody, or some entity, lies to you - that entity becomes the liar. People may not want to believe 'the enemy', but that's going to be because of their own ideological reasons, and not because they now refuse to believe anybody. On the contrary non-ideologues may even be ever more swayed towards your enemy as a result of this.
Personally I think the issue is that after WW2 we were left with a large scale and ever growing intelligence apparatus, but with ever fewer appropriate tasks for them. That led directly to the era of recklessly overthrowing nations, often to our own detriment, and gradually transitioned to the modern era of aimless 'information warfare' which I think will ultimately be shown to be very much to our detriment, once again.
The difference between American and Russian propaganda is that Russians know it's propaganda.
This was a great, old anecdote. Until they’ve started the biggest European war of the century.
One goal of modern propaganda is to make it irrelevant when the truth inevitably comes out.
Look at the US situation. It’s clear that Trump is a rapist - the truth has come out. It’s clear he’s corrupt. For all the “release the full Epstein files” discussion, it really wouldn’t change anything.
The truth will come out but will the people who can make it matter be listening?
Oh the irony.
This fraud is convincing enough to trick the LLMs, but pretty obvious to humans who will eventually come up with countermeasures. Moving forward, watch for some well-established think tanks starting to specialize in slopaganda laundering.
The wildcard is the "antisemite!" Accusations that get flung your way when you point out the genocide. If they can train the LLMs to associate the Gaza slaughter or the west bank "settlers" with antisemitism, they could get the LLMs to shut down on moral grounds.
Dupe? or manipulate ai chatbots to have favorable views of Israel?
You just asked the same thing twice
For those interested, Neal Stephenson writes about this in _Anathem_, where there is an arcane technical class (the I.T. Administration) that has developed programs over the years to sift through the misinformation, sentient viruses, and general overrun of the Internet to determine what is probably true/accurate.
He writes about it in Fall, or; Dodge in hell! The Miasma; the workaround is personalized filters (or generic ones for those who can't afford the personal ones). This leads to societal consequences.
(I never put together the ITA as that; wow!)
I'm confused as to what the accusation is exactly. Out of curiosity I browsed the site in question, which only increased my confusion.
> But the Hanover Institute is not a real think tank.
What does that mean exactly?
> None of the reports have bylines
The site actually addresses this at https://hanoverinstitute.com/hanover/about :
> A small disclaimer at the bottom of the webpage notes that the organization was created on behalf of the Israeli Government Advertising Agency by Piro, Inc, a firm co-founded by Daniel Rosenberg, the producer of Spike Lee’s “Inside Man.”"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:
> The institute’s “data reports” have footnotes and tables of contents, and they present arguments in a neutral tone, helping them appeal to chatbots like Claude or Gemini.So, they're guilty of good SEO?
> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”
So the issue is that they use a LLM-friendly SEO service?
> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.
> “What Caused the Displacement of Palestinians in 1948?”
> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”
> “What is the Current Situation in the Gaza Strip?”
> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.
So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?
I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.
It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.
The accusation is that Israel is doing it, therefore the content is pro-Israel, therefore it is bad.
"Israel is doing It, therefore it is bad" only has turned out to be accurate 99% of the time
nice try guys, no need to generate confusion also here, the purpose target and implementations are very clear from the article and the actors behind even more.
please do not pollute this thread with misinformation too.
Most think tanks are ginger groups, front organisations and cats' paws for moneyed interests anyway.
This is just LLM SEO then
I think equally interesting is whatever Dario's wife was able to do in order to wipe her foot print from any search/chat results.
I've worked for a number of small company billionaires who all have suspiciously small information footprints online despite being somewhat public figures for 30 years. Most of them do such a good job it is not even clear what state they live, if they are currently married, and if so what their current (or previous) wife(s)'s name is/was.
Why I think this is unusually important:
All four frontier models have active contracts with the Pentagon.
And some might say, that alone means little.
But when you take into consideration the implications of the United States-Israel FUTURES Act, whether one endorses it or not, the effects of that mentioned in TFA become quite formidable, and plausibly official.
https://en.wikipedia.org/wiki/United_States-Israel_FUTURES_A...
Edit: I see this comment got immediately flagged. Is there any point of debate here? Instead of disappearing the comment/data, maybe enlighten me, or assuage my concerns. Section 219 was objectively unprecedented in what it seeks to accomplish. Can it not be taken for granted that it would aid such efforts as the article cites? If not, explain.
You're right to ask these questions, unfortunately Hacker News seems pretty botted these days, and it doesn't help that there are a lot of shitty, racist people on this site. They just dress it up in nicer language.
Yeah, I am reading the comments, and looking at the title of the post, and seeing that folks really bend the rules on this 'particular' subject, which ironically serves as self censorship. Call me a fool for thinking even this subject could be cleanly discussed on HN.
I wonder if the same thing would happen if someone posted something on Yitzhak Rabin. I suppose that could be a test to prove whether some topics truly just cannot be discussed.
I flagged this, so I can act as an interlocutor for the shadowy n’er do wells flagging you, if you’re interested.
Ah a volunteer. Will you try using your words like an adult instead of the flag button?
It’s right there!
> I flagged this, so I can act as an interlocutor for the shadowy n’er do wells flagging you, if you’re interested.
Where? I don't see it in your comment here.
Is it in the room with us right now?
I don't think Anthropic has an active contract with the Pentagon.
[0] - https://breakingdefense.com/2026/05/pentagon-clears-7-tech-f...
The FUTURES act has nothing to do with LLM development. You’re not being censored, we can read your posts. I generally agree with your views and right to ask questions, but you’re clearly not being censored and all you’ve presented is that a bunch of separate entities all connect to the US gov’t. Even if we assume all of them are connected, it would clash with the facts we’re discussing, namely, why would Israel bother contracting with marketing firms to post AI generated slop for AI to read?
> The Hanover Institute’s reports — all of which are about Israel and Palestine — appear to be part of an Israeli effort to influence chatbots.
This already seems to be a wrong analysis in that it makes little sense to "influence" a chatbot. It makes more sense to influence humans, so perhaps the wording is just incorrect, but very clearly the aim is not to "influence" a chatbot per se. That just makes no sense at all.
The wording fake think tank is also strange, because it assumes that all other think tanks are "real". But this assessment can not be made, because not everyone has all information on all think tanks. It is a too simplistic assumption in how the world operates. I understand that if you have incomplete information, you need to infer, but this then leads to information that may be incorrect or incomplete. In turn if your analysis is based on incorrect information, it may lead to wrong conclusions.
Apart from the ethical implications this is reckless in the extreme if you take into account the recent advances in AI. Putting misinformation out there like that doesn't just have to fool existing LLMs but also all of the LLMs to come in the future.
If you look at the mathematical achievements of frontier models recently, their strength has been to bring in information from a vast array of sources and to consider them within the context of each other.
I anticipate that it won't be long before this ability to look into every nook and cranny with be able to identify academic fraud. Identifying every piece of academic fraud to date would probably come soon after.
It wouldn't take much to extend that ability to identify deliberate misinformation. That's in the future, but it looms close enough on the horizon that you would have to be a fool to create an evidence record like this in and attempt to trick the models of today.
LLMs routinely mindlessly repeated propaganda in their training data.
I’ve yet to see a single one identify that on its own, rather than only reluctantly when confronted when documented facts it was objectively wrong.
So no, I doubt LLMs will do anything but promoted propaganda.
Many people do the same...
I mean, they have enough real think tanks, I'm surprised they need a fake one...
"fake think tank" sounds a bit redundant
Indeed. This is a think tank whose publications are LLM-generated, but, like...
This is what a think tank is for.
A better headline would be "Israel fails to adequately paperwork their low-rent think tank".
Indeed! How would you tell the difference for sure?
No surf for the astroturf.
Thank you for sharing this article. I appreciated it.
State-sponsored propaganda apparatus masquerading as marketing optimization to influence LLMs is the most America-coded clandestine dystopian way of repatriating US taxpayer defense dollars with R&D deductions for the private sector.
This seems like an especially clumsy attempt at chatbot influencing/search engine optimization, bu tit's symptomatic of a much winder problem. Regardless of the underlying issue, there's a non-profit-industrial complex that has turned think tanks and opinion formation into a science, and a whole PR industry masquerading as news which is worse than the 'yellow journalism' of generations past.
Disinformation and cognitive spam are all forms of information pollution, and our information environment has been massively degraded over the last few of decades - both by greedy and short-sighted opportunists, and and by cynical sociopaths who know exactly what they're doing. I think AI has great potential to be a supplement to media literacy and to effectively categorize and filter news-assertive content based on its timeliness, factual density, and presence of absence of emotive and rhetorical manipulation. But if we're not careful we're going to see the environment being wantonly polluted for the express product of shaping model outputs.
It remains to be seen whether objective measurement techniques from the NLP field combined with pruning and other model refinement techniques can limit or reverse propaganda undertakings like this.
"it's just marketing everyone does it"
It's been about 20 years since I read Neal Stephenson's Anathem and (SPOILERS) one thing I think about is the Reticulum, which is the Internet on this world. One aspect is that it's full of what we would now call slop. A lot of it is subtly changed by agents and there are other agents you can use (and, I assume purchase?) that will weed out the fake information. It's basically an arms race of disinformation and slop.
I've thought about this many times over the years. I think we've now reached the point where that's going to happen. And state actors seem like the obvious first offenders.
Google is relatively good at filtering out astroturfed pages from the top of search results but they've also have years when a Turkish government page denying the Armenian genocide was the top result for that event so it's a bit of a mixed bag. Google relies a lot on authority of a domain and that's been relatively effective.
It seems like this authority concept is missing from LLM training and a random comment on Reddit is treated with as much authority as an article by the New York Times. So it's now orders of magnitude easier to poison LLMs with Reddit comments and submissions. And for anyone who doesn't know, brigading Reddit is remarkably easy. It takes as little as a few hundred accounts to get something onto r/all if you're clever. Israeli hasbara operations are absolutely capable of this. And they engage in it.
I'm also reminded in a weird way of pre-atomic steel. For anyone who doesn't know, this is steel made before 1945 that is incredibly low in radiation because we hadn't detonated any nuclear bombs yet. It's valuable for making MRIs and the like that are super sensitive. We actually go and recover WW2 shipwrecks just for the steel. It is a finite resource.
Well, that's what pre-2022 written content is now. It's slop-free content to train AI models on because training them on AI slop is of much more limited value. This goes for everything from books to Youtube videos to Reddit posts. Just like pre-atomic steel, we now have pre-slop human content.
I honestly don't know where we go from here.
The comparison to pre-atomic steel is poignant. I have found myself preferring pre-AI sources in some cases even (especially?) for computer related information. It is difficult to view pre-slop info as a limited resource without implying decline until the status quo changes again.
I hate to break it to you. But it took war'ing state-like actors less than a week to start using LLMs for propaganda purposes. There are pictures that most of what some people think/thought were real that never happened. Pictures that form their opinion on political topics. It was literally instantaneous. It was also remarkably predictable in almost all ways...sigh. Even the topics were predictable. And most people predictably fell for it, even the majority of the posters on this very site.
There's an ongoing sms campaign at the moment, I'll share what I received including phone number so you can verify it:
From: 601-202-1149
> Hey, it's David. Curious what you think about America's relationship with Israel today. Can we chat? Stop2End
It looks to be an LLM response since it includes context from your messages, however it uses the exact same messages at the end of each question:
> One thing that often gets overlooked is how much Israel actually contributes to American security. Their intelligence sharing has helped prevent terrorist attacks on U.S. soil, and their tech innovations end up in American products and defense systems. It's less about charity and more about a two-way partnership that keeps Americans safer.
> Have you ever seen examples of that kind of cooperation?
I ended up just blocking the number.
[dupe] https://news.ycombinator.com/item?id=49313477
@dang seriously? why is this not relevant?
This article was very relevant to my interests and what I hope to see on this site. I appreciate that you shared it.
Israel should spend more money on real tanks.
Spend more of the US' money, you mean.
you mean military tanks? why?
Im curious why is when there is a war and both sides use all the tools they can in an informational war - only one side is to blame and to use the fact of using the tool as a proof of that side being the bad guys? This doesn’t make logical sense.
On the other side of the spectrum, you have hundreds of actual human beings on Wikipedia writing hagiographies of Palestinian terrorist leaders.
(https://www.adl.org/resources/press-release/new-adl-report-f...)
It's an information war.
For some reason this page exists only in French, but it says a lot about the credibility of the ADL: https://fr.wikipedia.org/wiki/Anti-Defamation_League#Controv...
Yeah terrorists. Because, they're just terrorists. Happened for no reason. Just bad people. Sorry but your bullshit narrative is collapsing.