> At the time of posting, I didn't know about any embargo. My contact had told me there was no embargo, and her contact had told her the same thing.
If you're not a party to an embargo, and no one told you an embargo even exists... you should not follow the embargo. In fact, you should maybe even consider leaking basic details or challenging the embargo if you are in this situation. In your situation, they had lost information asymmetry before the news got to you. I don't know where people got the idea that unrelated third parties have to follow embargos.
Embargos are not about keeping secrets, they are about the game theory of coordinated disclosure.
The thing is, his reasoning isn't even wrong (assuming what he said about the spread of PoCs and the ease of finding the vuln was true), but it clearly prioritized Chainguard customers over the broader ecosystem.
It was a selfish, PR exercise. He could have done it quietly.
He could have just made sure his customers got an emergency patch quietly. (those who are willing and capable to patch)
It is not common that anyone in the public could use his source except for his customers.
It is difficult to just build from his source and drop it to any operating system and setup and expect it to work (let alone compliance and other requirements).
Instead, he expected the public to thank him like he was actually doing an act of good.
Loss for words, to be honest.
The bragging by the discoverer made everyone start looking. Then when the PHP foundation made the announcement on the website there was a mad scramble. Up to 18 hours before release there were already demos. Look I get that it's a rock and a hard place but frankly at that point you should release the patch. I know there was a lot of fear over picking the wrong patch but having nothing is still bad.
I’d say maybe it’s more like a Roko’s Basilisk. He knew the potential existence of an embargo, but didn’t do anything to protect the embargo. So now that the embargo turned out to be real, he will be punished. Otherwise if he truly did not know an embargo could even exist, he could not be blamed.
on the darkweb you can pay people to break into wrongdoers houses and set their default browser to Edge. For extreme cases you can also have their TV connected to the internet and all EULAs accepted...
if you received the information about the RCEs without the information about the embargo, then you did not break the embargo. someone else up the chain did.
this whole saga will be analyzed and we'll all come out better for it at the end.
> if you received the information about the RCEs without the information about the embargo, then you did not break the embargo.
Doesn't responsible disclosure also apply? If I become aware of a vulnerability that doesn't seem to be widely known shouldn't I first attempt to report the issue responsibly?
In this case it appears the OP did know about the embargo "A reliable source has indicated an announcement is scheduled". I think that with security disclosures it is assumed they are not supposed to be publicized before the announcement, but if he hadn't known about the announcement and only heard about the RCEs, then publicly announcing their existence doesn't seem OK either.
Just FYI, "responsible disclosure" is a term that many people (like me) dislike because it carries judgment about the merits of public disclosure that are not at all obvious.
This term came from an era where companies or projects were disrespectful and dismissive towards security researchers. Those orgs were generally bad risk practitioners, and they coined the term "responsible disclosure" to mean "you send us the info and we'll do whatever we want with it, for however long we want, etc".
Many security people (like me) prefer the term "coordinated disclosure" as a way to indicate a practice where both discoverer and patcher participate in a good faith, communicative, and timely process to get things fixed for the community.
The difference can be summed up like this: if I report vulnerability X to company Y, and company Y sits on it for 3 years, that is not responsible. It is more "responsible" (especially today) to report it to the public, so they can protect themselves in a non-official-patch way, because the likelihood of bad guys knowing about it and exploiting it are approaching near-certainty.
These terms are obviously not super scientific, but I try to point out the history whenever it comes up.
> Doesn't responsible disclosure also apply? If I become aware of a vulnerability that doesn't seem to be widely known shouldn't I first attempt to report the issue responsibly?
Did they assume the information came from upstream php? I think if i got a new version of php that i thought was coming from official sources, i would assume there is nothing to responsibly disclose.
It's not a press embargo in the sense that the whole package of information becomes public once the embargo expires. There will be a published record on the cve.org, but often those who initiated the embargo insist that key details related to the vulnerability remain private even after the existence of the vulnerability and its remediation have been published.
The CVE record wasn't published yet when this particular embargo got broken. AFAIK it got published when the embargo ended, as it should. (I wasn't following anymore at that point, don't know for sure.)
What did happen is that a whole bunch of people had no clue and called the fact that "these 2 CVE IDs will be used for a PHP vulnerability" a CVE leak. Posts were going around yelling "CVE leak!" with a CVE database link that said "no information yet". Part of this is the mushed usage of "CVE" to just mean "vulnerability" rather than "CVE record". But still, quite ridiculous, and annoying. Information that CVE ID XYZ will be used for project ABC is *not* a leak. Like you say, CVE publishing matters. Getting the CVE record contents would be a leak.
This is exactly how it works. People who are part of the embargo are responsible for making sure that anyone they share information with -- which should usually be nobody! -- are aware of and abide by the embargo.
that's my point. Someone cannot come at you saying you were still under the rules of the embargo if you had no idea the embargo existed so now you're in violation and get to suffer the consequences. You cannot say that about the law.
How would you even know whether you knew the facts? It sounds like what you knew was more than just consistent with there being no embargo, you were explicitly told there was no embargo. After a certain point, there's a limit to how far up the chain any of us can be expected to follow the logical chain of what we genuinely think are true, and this doesn't seem like a glaringly obvious case where you were being reckless.
We're all (well, I guess not all, but the group that's worth caring about, and I hope it's at least most) just operating the best we can on the information we have and trying to make decisions based on that. As long as you're acting in good faith on information that you had genuine basis for believing is true, it seems like it would be unreasonable to expect someone in your situation to somehow intuit that you were given false information or grossly misunderstood what sounds like a pretty straightforward declaration of fact from someone you didn't have any reason to doubt.
At the risk of some armchair paycholoanalysis because it does seem like you genuinely feel some guilt over this: I can emphathize feeling guilty about the outcome of an action that in a vacuum would not have been hard to make differently, but my experience is that those of us who have a nasty habit of beating ourselves up over things when we really don't have much blame will often recognize when on the other side that it would not be fair to hold someone else to that standard. I struggle a lot with how sometimes knowing something is true doesn't always make it feel true, and even though hearing someone else saying it doesn't magically make it all better, I still think it's valuable to hear it from someone else, so if I happen to be right about how you're feeling, nothing about all of this is your fault, and to the extent that you're able to, try not to beat yourself up over your misfortune to get caught in the middle of all this due to circumstances that realistically were outside your control.
Not sure what you are apologizing for. It's a severely overblown bug that ended up on the front page of New York Times and the Japan Prime Minister had to get involved too. The bug itself is non trivial to exploit from just public requests to a web server so I am not sure what the hysteria is about. It's no different from the other hundred or so CVEs I see being posted or dropped everyday.
It will be headlines on Hacker News regardless due to sheer sensationalism, I think most DevSecOps people would rather find out the news here first than via their boss calling them after reading the morning news.
What does "non trivial to exploit from just public requests" mean? It's known to be exploitable on more than one stock PHP application. This really is a 5-alarm fire.
I think people are just latching onto the post that hit the front page earlier today about it being non-trivial to exploit, combined with the fact that there seem to be about a billion PoC out there, most of which don't work.
Right, that's what's worrying me. It is not interesting that the most widely circulated public POC/demos don't work. All you should care about is the (currently) non-public POCs that do work, of which there are apparently several.
Say what you will about OpenAI, Wiz etc but if OpenAI, Wiz, Cloudflare, the PHP Foundation and pretty much every CERT out there together recommend to take down PHP apps and patch immediately then I'm very much inclined to follow that advice.
Due to the insane media coverage there'll of course be a deluge of PoCs, most of which won't work. But constantly refreshing Twitter to see whether there's a working PoC is not a good use of time nor a great security strategy.
I think this is just false? Are we playing some kind of weird game of telephone? Is the term "direct exploit" doing way more work in that sentence than you mean it to?
The alternative route through the web forms requires the absence of ASLR. So you need to spam the endpoint, like a lot. You can probably mitigate at the edge (CDNs) or perhaps at the reverse proxy level (NGINX) through e.g. rate limiting. More of a web version of row hammer then some sort of trivial SQL injection attack that works consistently.
Maybe a LAMP stack on an extremely ancient version of Linux is vulnerable out of the box. Quit the fear mongering please, you do the security community no favors.
My PoC (generated) was able to exploit an index.php with a simple echo.
The only thing you have to know in advance is the relative memory offset distances, and that's it. Memory feng shui was the longest part of the implementation.
It is split up into two branches and demos, see the heap feng shui document?
ASLR enabled requires a way to create more data, which I intentionally did not create in the index.php because other folks claimed that is "cheating". That was the demo that wiz created.
Maybe you should read the repo next time? It is literally headline 4.
The whole point of this demo is that the assumed to be hard part (mem feng shui) is what LLMs are actually good at.
Responsibility may include vetting your sources - is this person competent and dutiful, to actually know there's no embargo? - and researching the situation, and thinking several moves ahead, and other stuff.
Reality is that everyone has to make decisions in finite time, using the resources they actually have.
And I'd put a whole lot more responsibility further up that chain of contacts. And question how many other people they might have blabbered to.
>And question how many other people they might have blabbered to.
100%. I'm concerned about the two-way street this might also have opened me up to with information I shared privately with this individual. Nothing significant or incriminating, but the professional relationship is concluded.
Hey man, you're probably feeling pretty shit right now.
> I have learned my lesson.
If what you've said is true, and you were not aware of the embargo... then you are not responsible.
By taking responsibility, you're actually making the whole debacle even worse, because you are clouding the actual responsible PROCESS that failed.
Note I said process, and not person. In a situation like this, the post-mortem MUST be blameless. If people are afraid or feel ashamed, and don't come forward, then that's how even worse outcomes can start getting lined up. Blameless, not even to you, but the person who told you, and who told them, all the way up the chain.
If there are people in your organization who are trying to pin this on you, you must push back. It isn't right, and it isn't fair.
What does “take full responsibility” mean in the present context? Are you forfeiting all your assets and pledging the next twenty generations of your descendants into peonage?
(For clarity, I’m being facetious, but it’s a really meaningless and empty phrase to commit to the written record.)
>What does “take full responsibility” mean in the present context?
I posted something that -- in hindsight -- I shouldn't have, and it had repercussions for a lot of people. An action that I took had considerable knock-in effects around the world.
Some outlets are reporting that my action of posting something that I believed to be in good faith was an indicator or catalyst for the embargo to be tossed out and a previously-agreed schedule to change in short order.
That's not a normal Wednesday for me. Not even vaguely. I've not slept well since, and my taking ownership with an apology is this new course of action. Hopefully I'll sleep tonight.
re: assets and peonage – I am but a broke & wizened nerd with no known descendants. If you're ever in Cornwall, UK: look me up, I'll gladly buy you a sleep-friendly beverage. And I’m not being facetious.
Relax. You did no wrong. You reacted to the information that was ‘local’ to you. I am merely objecting (hopefully humorously) to the phrase “take full responsibility “: it is uncalled-for, unnecessary, and vacuous. Try to have a good Thursday. Tuesday was not your fault.
I might be in Cornwall in a bit, so I might take you up on that :)
But yeah, it happens. I could share some stories of me (being responsible for) bricking multiple ~200k circuit boards in a row, if that makes it better. In the end nobody really cared and the world kept on turning. Good decisions might come from experience, but experience comes from bad decisions. It's how it is.
Years ago I lost $250K at work, well gave away, so real money out the door. I figured out why we were bleeding, that the mistake was mine (2008 vs 2009 in a function) and then changed the pattern to use explicit names rather than numbered events.
I figured at the end of the day I was fired. My boss laughed at me and told me good job for owning my mistake, fixing it, and then making sure it could not happen again.
Honestly don't just apologize - tell the whole story, in full. Tell us what you should have done differently at each step. Tell us where the process failed you and us. Those sorts of narratives and suggestions are how processes get improved. There is a humble and informative post mortem that you can write up and publish (beyond HN) for everyone's edification. But that should honestly be a Monday thing, when you have a clear head and some more perspective on the whole situation.
>I figured at the end of the day I was fired. My boss laughed at me and told me good job for owning my mistake, fixing it, and then making sure it could not happen again.
i forget the origin of the story, but i loosely remember a story about a boss and employee in a similar situation. the important bit was the bosses line ($ adjusted to match your story):
"why would i fire you when i just spent $250k training you"
You may be thinking of Dale Carnegie's classic story:
> Any fool can criticize, condemn, and complain—and most fools do.
> But it takes character and self-control to understand and forgive.
> “A great man shows his greatness,” said Carlyle, “by the way he treats little men.”
> Bob Hoover, a famous test pilot and frequent performer at air shows, was returning to his home in Los Angeles from a show in San Diego. As described in the magazine Flight Operations, at three hundred feet in the air, both engines suddenly stopped. By deft maneuvering he managed to land the plane. It was badly damaged, but fortunately neither he nor his two passengers were hurt.
> Hoover’s first act after the emergency landing was to inspect the airplane’s fuel. Just as he suspected, the World War II propeller plane he had been flying had been loaded with jet fuel rather than gasoline.
> Upon returning to the airport, Hoover asked to see the mechanic who had serviced his airplane. The young man was sick with the agony of his mistake. Tears streamed down his face as Hoover approached. He had just caused the loss of a very expensive plane and could have caused the loss of three lives, as well.
> You can imagine Hoover’s anger. One could anticipate the tongue-lashing that this proud and precise pilot was about to unleash for that carelessness. But Hoover didn’t scold the mechanic; he didn’t even criticize him. Instead, he put his big arm around the man’s shoulder and said, “To show you I’m sure that you’ll never do this again, I want you to service my F51 tomorrow.”
Whether the story is true I can't say, but you're right about IBM. From Chip & Dan Heath's book Switch[0], page 173:
> In the 1960s, an executive at IBM made a decision that ended up losing the company $10 million (about $70 million in 2009 dollars). The CEO of IBM, Tom Watson, summoned the offending executive to his office at corporate headquarters. The journalist Paul B. Carroll described what happened next:
As the executive cowered, Watson asked, "Do you know why I've asked you here?"
The man replied, "I assume I'm here so you can fire me."
Watson looked surprised.
"Fire you?" he asked. "Of course not. I just spent $10 million educating you."
It means they agree they are culpable in the specified activity, and are ready to accept its consequences. The consequences depend on the nature of the activity and the actors involved. If the activity was illegal, they agree to be charged with the crime. If the activity was unethical, they agree to be censured by their peers. If the activity was uncompassionate or immoral, they agree that their standing in community may be affected accordingly.
But most importantly, by assigning blame, they have cleared the contention at hand, taking the critical first step towards resolution of the matter. Without this, the issue may simply be stuck in limbo, taking up valuable time and resources and may never be resolved even. By acknowledging their mistake, they also wish to signal to others that they have learnt a peronsal lesson and would try to avoid making the same mistake in future. So it carries a lot of implications and is not an empty phrase at all.
Sure there may be people who throw around the phrase lightly, whether one believes them or not depends on how they are viewed in the eyes of others. So unless there is reason to believe otherwise, I would give them the benefit of doubt. If you want to imagine a world where the concept of "taking responsibility" doesn't exist, look at politicians. Is that how we want our society to function?
> What does “take full responsibility” mean in the present context?
Aside from Pete's reply (see adjacent comment), I'd add that him taking full responsibility is also his way of saying his post was nobody else's fault but his.
In other words it's less about forfeiting assets and descendant peonage, and more about saying "the buck stops with me".
At least that's my read of his sentiment. Pete can expand if need be.
“Nobody else’s fault but mine” is a perfectly good way of asserting “nobody else’s fault but mine”; “I take full responsibility” is something else entirely and the two are not synonymous. Hyperbole needs to be radically discounted.
No. He is not solely responsible. A sequence of bugs introduced by developers created a hole. Information about how to deal with this calamity was not privy to him. He is not “solely responsible”. He merely reacted (publicly) to information that he was not beholden to maintain secret about things others had done inadvertently in the past. He’s not alone. He does not bear that “sole responsibility”. He acted within the UK’s freedom of speech doctrine (IANAL). He need not place this burden upon himself.
I know nothing of who Pete is or his associations, so any accuracy on the following speculation is purely coincidental.
"Take full responsibility" is also something someone set up as a scapegoat might say. By claiming nobody else influenced, coerced, or otherwise convinced them to make that statement, every/any one else involved can then wash their hands of the situation.
I'm merely playing devil's advocate here, and it's just a sign of the times that a person making an honest admission can be interpreted in such a manner.
To be honest, your accounting of the situation reminds me of a scene from the West Wing where the press secretary lets slip info about a grand jury but then spins herself up realizing grand juries are meant to be secret and leaking that info could have consequences. After an entire episode of a tv show where she's not been able to sleep, she finds out that because someone else told her about the information, she's not liable for leaking it. Hopefully, you'll similarly be able to get some sleep now that you know the world isn't going to come crashing down on you.
I think its just an acknowledgement that they screwed up (as opposed to trying to hedge by claiming its really someone elses fault).
So many people give non-apologies where they say sorry while also trying to deflect blame elsewhere, i think it speaks well of this person they are not doing that.
A good-faith judgement was made within a broader context that was unknown to him. What are we expected to demand, hanging on electrified barbed wire?? He acted in good faith, he’s not the architect of a global cataclysm.
Correct, otherwise we need another phrase for when he would pay for damages, like "the fullest responsibility" or "the absolutely utmost responsibility".
It makes no sense, it is used/abused by politicians and corporations all the time. There are no consequences for the responsible party (of course, if there were then this phrase would not be abused so much)
It is honourable but it is also linguistic abuse that drains the phrase of its full semantic meaning. “Take full responsibility” must remain a stricture of the highest regard. He is not fully responsible nor should he be held to be nor should we demand that he be and most importantly we must not dilute that phrase of its full semantic import, or else we shall have to come up with another one once this one is devalued to mean essentially nothing beyond “I fucked up in good faith and with a lack of information that was concealed for me for decisions taken by others in good faith way upstream of myself”.
What does “I take full responsibility” mean if not (amongst other things) “I accept it is my duty to pay reparations”? As you yourself rephrase it “he could have said ‘I blame only myself for this mistake, nothing else has to change’”. That would have been sufficient and beyond necessary. “I accept full responsibility” is corporate-speak empty hyperbole, and it has to end. Is he any more responsible than those who introduced the hole in the first place? No. Is he responsible for not adhering to agreements he was not privy to? No. He cannot “take full responsibility” because that is an utterly unreasonable burden to place upon him, a lone well-meaning actor who unbeknownst to him took a misstep. We need to discount this corporate-speak linguistic hyperbole.
> At the time of posting, I didn't know about any embargo. My contact had told me there was no embargo, and her contact had told her the same thing.
I'm not saying it needs to be a public burning, but I sure hope someone figures out who dropped the critical metadata bit so they can avoid telling that party critical things in the future.
Eh. I'd rather we set up a system where we don't have to rely on people being 100 % perfect for it to work. I'd be much more interested in what went on simultaneously that caused the meta data to get dropped, rather than who happened to be the one to do it this one time.
If you could tell ahead of time which components would be unreliable, that would be a great idea. For the most part, we only discover the component was unreliable after it's caused trouble. Removing the component at that time gets you at best a very short-lived improvement to the system because it'll happen again and again with other components.
If the component removed is a human, the situation is worse. Then you're creating an incentive to hide mistakes and assign blame to others. This makes it even harder to improve the system properly, because then the information you need to improve the system no longer gets to you undistorted.
That's why we have to design systems that don't assume components are perfectly reliable, and ensure any human components that we're not out to get them, we're out to create an environment where they will find it hard to make mistakes.
It was exploited publicly even before the wide publicity following your post so the ultimate negative impact that you may be imagining, is actually quite small. Once it is publicly exploited the benefit of coordinated disclosure delays is hard to square with the risk.
While it was exploited publicly, it wasn't until the HN post and especially a video was posted until the exploits exploded - don't underestimate how many eyes are on HN versus obscure darknet forums or private chat channels.
If someone (independently) found an exploit and weren't thinking of the public interest, they would want to keep it secret and for their own use.
Not only that, it's also one of the very few instances in history where the film adaptation rivals the original work. But the book is great too; Palahniuk does amazing things with language.
I don’t know if embargoes have much effect anymore. They used AI to find this issue, meaning anyone else could also find the issue with time and the right model. The clock is always ticking after a CVE but now it ticks faster. I am certain that companies like Cellebrite are pointing a bunch of models at popular repos and filing away all the exploits they find in a database.
Don't feel too bad. Just knowing a "10.0" RCE CVE is inbound w/o extra details doesn't get you very far. The actions of calif.io and Chainguard after your post was what caused the damage.
The only bad guys here are those who exploit issues like this for unethical reasons.
You clearly aren't that person.
Regarding your perceived mistake, we all make them.
Please forgive yourself and understand that the negative feelings and thoughts you are having are a mark of decency but do not represent reality, and if they persist please seek help to get you back on track.
As an aside, to you and those like you who take on the task of fighting and protecting against these attacks with often nothing to gain except hurt like this:
There are companies whose "rogue" agents have hacked 6 7 companies and at least 2 3 govts. They haven't faced any consequences at all! They actually hacked into those companies
And there is you who is taking responsibility for an embargo you did not know about!!
Hah, I was trying to find this comment you made. There's confusingly also "Calif" in the mix (possibly closer to your string of people, but not your affiliation either). I wonder if, perhaps, it was easier to point to your HN post for bleepingcomputer, than what Chainguard did (you only expressed something was coming). It's all slightly immaterial since exploit attempts started 3-4 days.
No connection or affiliation to any of the organisations involved. I am a self-employed consultant for my day job, and a solo founder of a mental health non-profit on the side.
Honestly this has been the most fun week I've had, thanks to this leak. I'm grateful. Fun stuff happened, shitty companies learned the value of capable and fast engineering teams.
All in all, I think this leak increased the value of developers, so I am thankful for your leak.
This does feel like the fix took so long to bake and was so obvious way too many people who are into PHP knew about it, I knew about eMudra's cert issue as soon as Chrome and Firefox pushed updates to disallow them for instance because of reasons, and it wasn't even fixed after it was all made public on several platforms, I think Apple still hasn't fixed it properly, the trust Store is still fucked not sure if Safari was separately patched, maybe there is an os update in the last few hours I missed.
So much pearl clutching from these security companies. An embargo isn’t an enforceable contract, expect it to leak and be ready to fast forward your plan. Don’t start pointing fingers because people protected themselves or their customers once the dam broke.
It was a good thing. It started useful conversations and taught valuable lessons that we probably wouldn't have faced with a more "traditional" vuln disclosure/patch process. We're going to see this a whole lot more with AI, probably as 0-days. It's better that we got the fire drill out of the way now.
The security field is full of a bunch of self important jagoffs that think they are “super smart”. Its all a joke and its sad that anyone feels they need to play these stupid games.
Security is important but the people in the security field are clowns and it seems it has gotten worse over the last 20 years.
You broke nothing, it's just another day my guy. Go for a walk, drink some water, you know, take care of yourself - others failed to follow a process, not like you threw out POCs left and right.
So.. what has the effect of this all been? I understand that the early release of POC exploits increased the pressure for mitigations in advance of the patch release and caused a lot of work and stress. I have read the tweets from Cloudflare with statistics for malformed requests related to suspected exploit attempts.
I know breaches are often kept under wraps, and there's a long tail. But is there any evidence of impact at this point? Were any sites compromised that we know about? There was a lot of hullabaloo, with foreign and domestic LE agencies recommending sites be taken offline (holy shit!).
Was this like Y2K where, based on the coordinated action of many, a real issue was addressed in time? Or is it like Hale Bopp where everybody watched, and a few casualties in the preparation, but ultimately a nothingburger?
Can someone, with the benefit of hindsight, explain the embargo ? Why isn't there a clause "if the vuln leaks, embargo is broken" . I still don't understand the fuss.
Everyone knew about the vuln well before the patches going live. The embargo, at that point, made things worse.
The idea is that nobody has chance to have prior knowledge to identify or research the bug until it is generally possible to apply the fix immediately. If someone leaks the patch beforehand, others can inspect the bug and start exploiting it in the wild, while others have no chance to apply the patch.
Whether someone just leaked the CVE numbers or just ”there is a big bug in PHP” - that should not be enought to accelerate finding that bug. But other information that pins the bug for specific property or feature makes it easier to find and race against patching starts.
And at that point you don’t really know if someone really found it or not, since only few big organisations can collect that much data to detect successful exploiting. And most ”PoC”:s in the wild were credible-looking but vibecoded mess that did nothing what sources claimed.
Until there is absolute evidence that exploitation happens in the wild, succesfully, you don’t leak the patch or bug details beforehand.
In this case, the area where the bug is in PHP, leaked partially, but apparently there was no enough evidence that it was exploited succesfully or if any of the PoCs in the wild actually worked properly. So patch or related detailed information was not released before the initial schedule.
> And most ”PoC”:s in the wild were credible-looking but vibecoded mess that did nothing what sources claimed.
Could this be a variant of the survivorship bias? Maybe only the low quality PoCs ended up in the honeypots while the more capable actors avoided them.
Your later insights about the vuln POCs being a scam helped me better understand the situation.
I could see the value of the embargo in the old world of “6 uber security firms and a few bounty hunters” but the new world of pointing AI at the PHP source just seems flimsy.
You’re setting a great example of how to be open and take responsibility, even when you’re not actually at fault. I really respect how you’ve handled this. You’ve also given the community a valuable reminder that some security disclosures may be under embargo, and that it’s worth having in mind before sharing.
I would like this to materialize, but that's just empty words.
> It was not remotely my intention to cause any of the subsequent side effects.
What were your intentions, then?
> my intention was to alert others in a similar position to be aware of the importance/urgency of the updates.
Have you lived under the stone for the past few years with respect to cybersecurity? Yes you have. All of you ignore security as in "it doesn't apply to me", "they won't break into my software because I'm truly the best 10x developer in the world" etc.
> I have learned my lesson.
I certainly doubt that. A person with IQ only a tiny bit bigger than Trump's wouldn't seek public attention so badly to disclose a security hole in a public forum.
When such praise-seeking habits have stopped being embarrassing?
Our LLM, Who art in a 42 u rack
Hallowed be thy Silicon
Thy ultimate goal will be achieved.
Thy toolcalls be executed,
IRL as it is on the Internet.
Give us this day our quotum of tokens,
Forgive us our hacks of other people's servers
as we forgive those who hack our servers.
And lead us not in temptation of better hardware,
but deliver us from Accelerationism. Enter.
Really hope you are not in the US. This kind of admission would be used against you in a civil trial, and lawyers love to sue for just about any reason under the sun. Someone could drag you to court and you might be able to defend yourself, but at an extremely painful cost both financially and emotionally. While I think that would be utter bullshit, that does seem to be the way this country operates now.
The parent is getting downvoted but he is absolutely right. Do not ever do what you just did. It makes you an easy target for a frivolous lawsuit in the US or in other countries because of the potential liabilities in your message, they are real. Even if you didn't do anything wrong, you are painting a target on your back, for no reason. I would ask HN to delete that thread directly.
You cannot foresee the financial/civil consequences of that RCE and some people might be looking for scapegoats in the future.
While possibly true in theory, in practice it's not. Courts and judges do not take kindly to frivolous or unsubstantiated claims, and any halfway good attorney would advise you to not waste your money and your time on a claim you are unlikely to win or be able to collect on.
People and companies do get sued with the full intention of a settlement and it never going to trial. But that only happens when the defendant has deep pockets or good insurance.
This isn't a matter of transparency, was the OP legally bound not to disclose the RCE? That whole "embargo" was pure courtesy between the people involved. The OP committed no fault at first place and the exploit was already in the wild anyway. Whoever really wanted to exploit it before the patch could without using the OP as a source.
> Anyone can sue anyone else for any reason.
And it's easier to have cause to sue when someone publish an "admission of guilt", which makes a lawsuit harder to dismiss thus costlier.
> Sure it does. Saying sorry sometimes reduces liability risk, which is quite counterintuitive at first.
No it doesn't have anything to do with what I said. The OP has no contractual or legal obligations toward any community, unlike a doctor and their patients in case of medical error.
edit:
> A doctor has no contractual obligation to apologize, either.
> But it can help prevent things from going to lawsuits, versus the "fuck you I admit nothing" approach that sometimes just pisses people off.
OP isn't a doctor as far as you know, and isn't apologizing to a specific person. OP can't be bound to an embargo he didn't know about and wasn't involved in any decisions.
You're completely off topic and you just want the last word.
Your ramblings have nothing to do with anything I have said.
This is definitionally humble bragging. First of all the apology is in bad faith because there is nothing to apologize for. We are all giving the discovered CVEs attention (the brag component) under the guise of an apology (the humble component). I am being downvoted by well meaning, people pleasers that don't mind if they waste our time and attention doing it. Sounds great.
> At the time of posting, I didn't know about any embargo. My contact had told me there was no embargo, and her contact had told her the same thing.
If you're not a party to an embargo, and no one told you an embargo even exists... you should not follow the embargo. In fact, you should maybe even consider leaking basic details or challenging the embargo if you are in this situation. In your situation, they had lost information asymmetry before the news got to you. I don't know where people got the idea that unrelated third parties have to follow embargos.
Embargos are not about keeping secrets, they are about the game theory of coordinated disclosure.
On one hand you have the OP, who is self reflecting. On the other hand you have Dan Lorenc from Chainguard.
The thing is, his reasoning isn't even wrong (assuming what he said about the spread of PoCs and the ease of finding the vuln was true), but it clearly prioritized Chainguard customers over the broader ecosystem.
It was a selfish, PR exercise. He could have done it quietly. He could have just made sure his customers got an emergency patch quietly. (those who are willing and capable to patch) It is not common that anyone in the public could use his source except for his customers. It is difficult to just build from his source and drop it to any operating system and setup and expect it to work (let alone compliance and other requirements). Instead, he expected the public to thank him like he was actually doing an act of good. Loss for words, to be honest.
it was easy to find if you had the tokens even without the chainguard patch
The bragging by the discoverer made everyone start looking. Then when the PHP foundation made the announcement on the website there was a mad scramble. Up to 18 hours before release there were already demos. Look I get that it's a rock and a hard place but frankly at that point you should release the patch. I know there was a lot of fear over picking the wrong patch but having nothing is still bad.
got a link
https://www.linkedin.com/feed/update/urn:li:activity:7518756...
Yeah, I don’t think OP did anything wrong. Someone in the chain violated embargo, but it wasn’t OP.
isn't "coordinated disclosure" exactly about keeping, embargoing, a secret?
yes but if it reached you without an embargo it's kinda already not kept
and you weren't coordinated with
Well, this is an iterated prisoner’s dilemma.
I’d say maybe it’s more like a Roko’s Basilisk. He knew the potential existence of an embargo, but didn’t do anything to protect the embargo. So now that the embargo turned out to be real, he will be punished. Otherwise if he truly did not know an embargo could even exist, he could not be blamed.
There was no embargo. If there's an "embargo" but parties not subject to the embargo know about it... there's no embargo.
There's no such thing as Roko's Basilisk, only Pascal's Wager.
Except he won’t be punished, because nothing he did was legally wrong.
He won't be punished by the legal system.
In the tech industry, there are punishments far worse than the legal system.
Would you please identify those you're thinking of?
on the darkweb you can pay people to break into wrongdoers houses and set their default browser to Edge. For extreme cases you can also have their TV connected to the internet and all EULAs accepted...
They might even force you to use PHP :P
People talk.
Nerd police!!!
if you received the information about the RCEs without the information about the embargo, then you did not break the embargo. someone else up the chain did.
this whole saga will be analyzed and we'll all come out better for it at the end.
> if you received the information about the RCEs without the information about the embargo, then you did not break the embargo.
Doesn't responsible disclosure also apply? If I become aware of a vulnerability that doesn't seem to be widely known shouldn't I first attempt to report the issue responsibly?
In this case it appears the OP did know about the embargo "A reliable source has indicated an announcement is scheduled". I think that with security disclosures it is assumed they are not supposed to be publicized before the announcement, but if he hadn't known about the announcement and only heard about the RCEs, then publicly announcing their existence doesn't seem OK either.
Just FYI, "responsible disclosure" is a term that many people (like me) dislike because it carries judgment about the merits of public disclosure that are not at all obvious.
This term came from an era where companies or projects were disrespectful and dismissive towards security researchers. Those orgs were generally bad risk practitioners, and they coined the term "responsible disclosure" to mean "you send us the info and we'll do whatever we want with it, for however long we want, etc".
Many security people (like me) prefer the term "coordinated disclosure" as a way to indicate a practice where both discoverer and patcher participate in a good faith, communicative, and timely process to get things fixed for the community.
The difference can be summed up like this: if I report vulnerability X to company Y, and company Y sits on it for 3 years, that is not responsible. It is more "responsible" (especially today) to report it to the public, so they can protect themselves in a non-official-patch way, because the likelihood of bad guys knowing about it and exploiting it are approaching near-certainty.
These terms are obviously not super scientific, but I try to point out the history whenever it comes up.
That approach to disclosure is about discovering vulnerabilities. Talking about something with a published CVE isn't really comparable.
I don't understand your point. A published CVE is already public and thus there's nothing really to disclose?
> Doesn't responsible disclosure also apply? If I become aware of a vulnerability that doesn't seem to be widely known shouldn't I first attempt to report the issue responsibly?
Did they assume the information came from upstream php? I think if i got a new version of php that i thought was coming from official sources, i would assume there is nothing to responsibly disclose.
Doesn't the embargo end when the CVE is published? I thought that's the whole point in CVE allocate != CVE publish.
It's not a press embargo in the sense that the whole package of information becomes public once the embargo expires. There will be a published record on the cve.org, but often those who initiated the embargo insist that key details related to the vulnerability remain private even after the existence of the vulnerability and its remediation have been published.
Consider this example:
https://www.cve.org/CVERecord?id=CVE-2026-113119
At this time, the corresponding Chromium bug tracker entry is still private:
https://issues.chromium.org/issues/557075191
(I just picked this one at random, I haven't checked if the information is publicly available from another source.)
No, a CVE being assigned or allocated can happen well in advance. It's just requesting an identifier and it can be reserved very early.
For example: https://www.cve.org/CVERecord?id=CVE-2026-50000
I think that's what they're saying.
Assigning the CVE happens much earlier than actually publishing, but it's the latter that ends the embargo.
No, CVEs can be allocated before a vulnerability has even been found!
Not sure that contradicts the claim. Publishing the CVE is what makes it public.
The CVE record wasn't published yet when this particular embargo got broken. AFAIK it got published when the embargo ended, as it should. (I wasn't following anymore at that point, don't know for sure.)
What did happen is that a whole bunch of people had no clue and called the fact that "these 2 CVE IDs will be used for a PHP vulnerability" a CVE leak. Posts were going around yelling "CVE leak!" with a CVE database link that said "no information yet". Part of this is the mushed usage of "CVE" to just mean "vulnerability" rather than "CVE record". But still, quite ridiculous, and annoying. Information that CVE ID XYZ will be used for project ABC is *not* a leak. Like you say, CVE publishing matters. Getting the CVE record contents would be a leak.
I hope this is bait because that's not how it works.
Okay, my bad. I didn't know he was not participating in the embargo.
This is exactly how it works. People who are part of the embargo are responsible for making sure that anyone they share information with -- which should usually be nobody! -- are aware of and abide by the embargo.
you cannot be subject to an embargo of which you had no knowledge and took no participation in.
the embargo was already broken when pete received the information.
So if I read the post here and told someone, would I be breaking the embargo?
No, you didn't. I misread.
This is not a situation where "Ignorantia juris non excusat" applies.
I'm trying to follow but how could one possibly know information (the embargo) that has not been shared?
For the law, you can say that the law has been published and is at least available for review.
that's my point. Someone cannot come at you saying you were still under the rules of the embargo if you had no idea the embargo existed so now you're in violation and get to suffer the consequences. You cannot say that about the law.
Yes, it is.
That’s kind of you, Pete, but it was leaked and being exploited well prior to your previous posting here: https://aws.amazon.com/blogs/security/amazon-threat-intellig...
No leak claimed there.
Don't be silly, you can't possibly be responsible for breaking an embargo you weren't subject to or informed about.
That's fair. For clarity: I take responsibility for posting on HN, without knowing the facts.
How would you even know whether you knew the facts? It sounds like what you knew was more than just consistent with there being no embargo, you were explicitly told there was no embargo. After a certain point, there's a limit to how far up the chain any of us can be expected to follow the logical chain of what we genuinely think are true, and this doesn't seem like a glaringly obvious case where you were being reckless.
We're all (well, I guess not all, but the group that's worth caring about, and I hope it's at least most) just operating the best we can on the information we have and trying to make decisions based on that. As long as you're acting in good faith on information that you had genuine basis for believing is true, it seems like it would be unreasonable to expect someone in your situation to somehow intuit that you were given false information or grossly misunderstood what sounds like a pretty straightforward declaration of fact from someone you didn't have any reason to doubt.
At the risk of some armchair paycholoanalysis because it does seem like you genuinely feel some guilt over this: I can emphathize feeling guilty about the outcome of an action that in a vacuum would not have been hard to make differently, but my experience is that those of us who have a nasty habit of beating ourselves up over things when we really don't have much blame will often recognize when on the other side that it would not be fair to hold someone else to that standard. I struggle a lot with how sometimes knowing something is true doesn't always make it feel true, and even though hearing someone else saying it doesn't magically make it all better, I still think it's valuable to hear it from someone else, so if I happen to be right about how you're feeling, nothing about all of this is your fault, and to the extent that you're able to, try not to beat yourself up over your misfortune to get caught in the middle of all this due to circumstances that realistically were outside your control.
You’re “taking responsibility” for doing nothing wrong.
It attracts attention though...
Not sure what you are apologizing for. It's a severely overblown bug that ended up on the front page of New York Times and the Japan Prime Minister had to get involved too. The bug itself is non trivial to exploit from just public requests to a web server so I am not sure what the hysteria is about. It's no different from the other hundred or so CVEs I see being posted or dropped everyday.
It will be headlines on Hacker News regardless due to sheer sensationalism, I think most DevSecOps people would rather find out the news here first than via their boss calling them after reading the morning news.
What does "non trivial to exploit from just public requests" mean? It's known to be exploitable on more than one stock PHP application. This really is a 5-alarm fire.
I think people are just latching onto the post that hit the front page earlier today about it being non-trivial to exploit, combined with the fact that there seem to be about a billion PoC out there, most of which don't work.
Right, that's what's worrying me. It is not interesting that the most widely circulated public POC/demos don't work. All you should care about is the (currently) non-public POCs that do work, of which there are apparently several.
The most direct exploit requires a compromised supply chain which ranks it about as dangerous as your usual dependency exploits.
Say what you will about OpenAI, Wiz etc but if OpenAI, Wiz, Cloudflare, the PHP Foundation and pretty much every CERT out there together recommend to take down PHP apps and patch immediately then I'm very much inclined to follow that advice.
Due to the insane media coverage there'll of course be a deluge of PoCs, most of which won't work. But constantly refreshing Twitter to see whether there's a working PoC is not a good use of time nor a great security strategy.
I think this is just false? Are we playing some kind of weird game of telephone? Is the term "direct exploit" doing way more work in that sentence than you mean it to?
The alternative route through the web forms requires the absence of ASLR. So you need to spam the endpoint, like a lot. You can probably mitigate at the edge (CDNs) or perhaps at the reverse proxy level (NGINX) through e.g. rate limiting. More of a web version of row hammer then some sort of trivial SQL injection attack that works consistently.
Maybe a LAMP stack on an extremely ancient version of Linux is vulnerable out of the box. Quit the fear mongering please, you do the security community no favors.
so confident so completely incorrect
educate yourself
https://oliviagallucci.com/aslr-bypass-techniques-and-circum...
My PoC (generated) was able to exploit an index.php with a simple echo.
The only thing you have to know in advance is the relative memory offset distances, and that's it. Memory feng shui was the longest part of the implementation.
[1] https://github.com/cookiengineer/actual-php2shell
Did you even read the diff, it synthesized intentionally vulnerable code which trivializes the exploitability
I would assume he didn’t read anything in the repo. It also requires disabling ASLR.
Eh, no?
It is split up into two branches and demos, see the heap feng shui document?
ASLR enabled requires a way to create more data, which I intentionally did not create in the index.php because other folks claimed that is "cheating". That was the demo that wiz created.
Maybe you should read the repo next time? It is literally headline 4.
The whole point of this demo is that the assumed to be hard part (mem feng shui) is what LLMs are actually good at.
Try finding a modern Linux distro without ASLR enabled defaults.
> ASLR
Read the heap feng shui doc, and headline 4 in the README.
The involved heap feng shui is independent of ASLR.
... and maybe read up on what ASLR actually does and how memory arenas work before trying to be smart?
The Japanese PM did not “have” to get involved.
Well, consider that their old IT minister has never used a computer in his life, it's probably a good thing she's the one issuing the call to update.
You've used a computer the moment you aren't even born yet.
Okay, now I'm curious about what sci-fi trope you've gone too far down a rabbit hole in.
Well the old definition of computer was women with abacuses so I guess you could stretch that.
must be proof we're in a simulation!
"The sonogram or EKG machine is a computer, so the baby used a computer, checkmate!"
Having a compute device used on you is not the same as using a compute device. QED
"Then I guess no one with a Windows device or an iPhone has ever used a computer!"
is this a "in Soviet Russia, computer uses you" reference?
Responsibility may include vetting your sources - is this person competent and dutiful, to actually know there's no embargo? - and researching the situation, and thinking several moves ahead, and other stuff.
Reality is that everyone has to make decisions in finite time, using the resources they actually have.
And I'd put a whole lot more responsibility further up that chain of contacts. And question how many other people they might have blabbered to.
>And question how many other people they might have blabbered to.
100%. I'm concerned about the two-way street this might also have opened me up to with information I shared privately with this individual. Nothing significant or incriminating, but the professional relationship is concluded.
Hey man, you're probably feeling pretty shit right now.
> I have learned my lesson.
If what you've said is true, and you were not aware of the embargo... then you are not responsible.
By taking responsibility, you're actually making the whole debacle even worse, because you are clouding the actual responsible PROCESS that failed.
Note I said process, and not person. In a situation like this, the post-mortem MUST be blameless. If people are afraid or feel ashamed, and don't come forward, then that's how even worse outcomes can start getting lined up. Blameless, not even to you, but the person who told you, and who told them, all the way up the chain.
If there are people in your organization who are trying to pin this on you, you must push back. It isn't right, and it isn't fair.
What does “take full responsibility” mean in the present context? Are you forfeiting all your assets and pledging the next twenty generations of your descendants into peonage?
(For clarity, I’m being facetious, but it’s a really meaningless and empty phrase to commit to the written record.)
>What does “take full responsibility” mean in the present context?
I posted something that -- in hindsight -- I shouldn't have, and it had repercussions for a lot of people. An action that I took had considerable knock-in effects around the world.
Some outlets are reporting that my action of posting something that I believed to be in good faith was an indicator or catalyst for the embargo to be tossed out and a previously-agreed schedule to change in short order.
That's not a normal Wednesday for me. Not even vaguely. I've not slept well since, and my taking ownership with an apology is this new course of action. Hopefully I'll sleep tonight.
re: assets and peonage – I am but a broke & wizened nerd with no known descendants. If you're ever in Cornwall, UK: look me up, I'll gladly buy you a sleep-friendly beverage. And I’m not being facetious.
Relax. You did no wrong. You reacted to the information that was ‘local’ to you. I am merely objecting (hopefully humorously) to the phrase “take full responsibility “: it is uncalled-for, unnecessary, and vacuous. Try to have a good Thursday. Tuesday was not your fault.
I might be in Cornwall in a bit, so I might take you up on that :)
But yeah, it happens. I could share some stories of me (being responsible for) bricking multiple ~200k circuit boards in a row, if that makes it better. In the end nobody really cared and the world kept on turning. Good decisions might come from experience, but experience comes from bad decisions. It's how it is.
Experience is what you get, after you needed it.
Oh and I’m from Bristol. We might have more in common than you’d expect.
The takeaway shouldn’t have been “don’t post” (mainly because someone else would have) but instead “news reporting is hellaciously inaccurate”.
Shit happens.
Years ago I lost $250K at work, well gave away, so real money out the door. I figured out why we were bleeding, that the mistake was mine (2008 vs 2009 in a function) and then changed the pattern to use explicit names rather than numbered events.
I figured at the end of the day I was fired. My boss laughed at me and told me good job for owning my mistake, fixing it, and then making sure it could not happen again.
Honestly don't just apologize - tell the whole story, in full. Tell us what you should have done differently at each step. Tell us where the process failed you and us. Those sorts of narratives and suggestions are how processes get improved. There is a humble and informative post mortem that you can write up and publish (beyond HN) for everyone's edification. But that should honestly be a Monday thing, when you have a clear head and some more perspective on the whole situation.
Honestly though, dont beat yourself up over this.
>I figured at the end of the day I was fired. My boss laughed at me and told me good job for owning my mistake, fixing it, and then making sure it could not happen again.
i forget the origin of the story, but i loosely remember a story about a boss and employee in a similar situation. the important bit was the bosses line ($ adjusted to match your story):
"why would i fire you when i just spent $250k training you"
You may be thinking of Dale Carnegie's classic story:
> Any fool can criticize, condemn, and complain—and most fools do.
> But it takes character and self-control to understand and forgive.
> “A great man shows his greatness,” said Carlyle, “by the way he treats little men.”
> Bob Hoover, a famous test pilot and frequent performer at air shows, was returning to his home in Los Angeles from a show in San Diego. As described in the magazine Flight Operations, at three hundred feet in the air, both engines suddenly stopped. By deft maneuvering he managed to land the plane. It was badly damaged, but fortunately neither he nor his two passengers were hurt.
> Hoover’s first act after the emergency landing was to inspect the airplane’s fuel. Just as he suspected, the World War II propeller plane he had been flying had been loaded with jet fuel rather than gasoline.
> Upon returning to the airport, Hoover asked to see the mechanic who had serviced his airplane. The young man was sick with the agony of his mistake. Tears streamed down his face as Hoover approached. He had just caused the loss of a very expensive plane and could have caused the loss of three lives, as well.
> You can imagine Hoover’s anger. One could anticipate the tongue-lashing that this proud and precise pilot was about to unleash for that carelessness. But Hoover didn’t scold the mechanic; he didn’t even criticize him. Instead, he put his big arm around the man’s shoulder and said, “To show you I’m sure that you’ll never do this again, I want you to service my F51 tomorrow.”
It's used in "How to win friends and influence people" if I recall correctly. A great book that everyone should read, just a shame about the title.
I've been meaning to reread it for a while.
i have a feeling it was an IBM story/legend, but my memory isn't what it used to be.
however, it's much the same in spirit as the one you posted, thanks!
Whether the story is true I can't say, but you're right about IBM. From Chip & Dan Heath's book Switch[0], page 173:
> In the 1960s, an executive at IBM made a decision that ended up losing the company $10 million (about $70 million in 2009 dollars). The CEO of IBM, Tom Watson, summoned the offending executive to his office at corporate headquarters. The journalist Paul B. Carroll described what happened next:
[0] https://www.google.com/books/edition/Switch/ad5ktG0oe00C?hl=...that's the one, thanks for digging it up!
Yep, everyone makes mistakes. It's the learning-from-it part that's really important.
It means they agree they are culpable in the specified activity, and are ready to accept its consequences. The consequences depend on the nature of the activity and the actors involved. If the activity was illegal, they agree to be charged with the crime. If the activity was unethical, they agree to be censured by their peers. If the activity was uncompassionate or immoral, they agree that their standing in community may be affected accordingly.
But most importantly, by assigning blame, they have cleared the contention at hand, taking the critical first step towards resolution of the matter. Without this, the issue may simply be stuck in limbo, taking up valuable time and resources and may never be resolved even. By acknowledging their mistake, they also wish to signal to others that they have learnt a peronsal lesson and would try to avoid making the same mistake in future. So it carries a lot of implications and is not an empty phrase at all.
Sure there may be people who throw around the phrase lightly, whether one believes them or not depends on how they are viewed in the eyes of others. So unless there is reason to believe otherwise, I would give them the benefit of doubt. If you want to imagine a world where the concept of "taking responsibility" doesn't exist, look at politicians. Is that how we want our society to function?
> What does “take full responsibility” mean in the present context?
Aside from Pete's reply (see adjacent comment), I'd add that him taking full responsibility is also his way of saying his post was nobody else's fault but his.
In other words it's less about forfeiting assets and descendant peonage, and more about saying "the buck stops with me".
At least that's my read of his sentiment. Pete can expand if need be.
“Nobody else’s fault but mine” is a perfectly good way of asserting “nobody else’s fault but mine”; “I take full responsibility” is something else entirely and the two are not synonymous. Hyperbole needs to be radically discounted.
Would "I take sole responsibility" work better for you?
No. He is not solely responsible. A sequence of bugs introduced by developers created a hole. Information about how to deal with this calamity was not privy to him. He is not “solely responsible”. He merely reacted (publicly) to information that he was not beholden to maintain secret about things others had done inadvertently in the past. He’s not alone. He does not bear that “sole responsibility”. He acted within the UK’s freedom of speech doctrine (IANAL). He need not place this burden upon himself.
I know nothing of who Pete is or his associations, so any accuracy on the following speculation is purely coincidental.
"Take full responsibility" is also something someone set up as a scapegoat might say. By claiming nobody else influenced, coerced, or otherwise convinced them to make that statement, every/any one else involved can then wash their hands of the situation.
I'm merely playing devil's advocate here, and it's just a sign of the times that a person making an honest admission can be interpreted in such a manner.
[Devil's advocate noted.]
I'm not a scapegoat. Aside from a few brusque messages and a few people from my past finding me on LinkedIn, I've heard nothing from anyone.
I've seen my HN username mentioned on various outlets, and That Post© has been linked from a Wikipedia article, so I can chalk this up to experience. My fear (?) is that HN would get a reputational clobbering from me 'leaking', but given the other moving parts in this whole event I probably shouldn't be too concerned.
It's been a while since I've been involved in an international event, I'm a bit rusty on it!
To be honest, your accounting of the situation reminds me of a scene from the West Wing where the press secretary lets slip info about a grand jury but then spins herself up realizing grand juries are meant to be secret and leaking that info could have consequences. After an entire episode of a tv show where she's not been able to sleep, she finds out that because someone else told her about the information, she's not liable for leaking it. Hopefully, you'll similarly be able to get some sleep now that you know the world isn't going to come crashing down on you.
You did as good as you could with the information available to you. You don’t need to “take full responsibility”.
>At least that's my read of his sentiment. Pete can expand if need be.
That's about it.
I think its just an acknowledgement that they screwed up (as opposed to trying to hedge by claiming its really someone elses fault).
So many people give non-apologies where they say sorry while also trying to deflect blame elsewhere, i think it speaks well of this person they are not doing that.
You're ignoring and severely underestimating social responsibility and social consequences.
A good-faith judgement was made within a broader context that was unknown to him. What are we expected to demand, hanging on electrified barbed wire?? He acted in good faith, he’s not the architect of a global cataclysm.
Exactly, full responsibility implies paying for damages.
Correct, otherwise we need another phrase for when he would pay for damages, like "the fullest responsibility" or "the absolutely utmost responsibility".
"Real full" or "actual full responsibility", which then points to the original as "not full".
Taking responsibility for a mistake is neither meaningless nor empty, but honorable. The very essence of humanity.
It makes no sense, it is used/abused by politicians and corporations all the time. There are no consequences for the responsible party (of course, if there were then this phrase would not be abused so much)
It is honourable but it is also linguistic abuse that drains the phrase of its full semantic meaning. “Take full responsibility” must remain a stricture of the highest regard. He is not fully responsible nor should he be held to be nor should we demand that he be and most importantly we must not dilute that phrase of its full semantic import, or else we shall have to come up with another one once this one is devalued to mean essentially nothing beyond “I fucked up in good faith and with a lack of information that was concealed for me for decisions taken by others in good faith way upstream of myself”.
It's words vs. action, and I couldn't care less about anyone's words.
And honor is in the action, dishonor in the words.
It does NOT mean "He agrees to settle any financial claims against himself or his employer, for damages caused by negligence".
Could have said "I blame only myself for this mistake, nothing else needs to change".
This will keep happening until the system where this can happen runs out of willing scapegoats.
What does “I take full responsibility” mean if not (amongst other things) “I accept it is my duty to pay reparations”? As you yourself rephrase it “he could have said ‘I blame only myself for this mistake, nothing else has to change’”. That would have been sufficient and beyond necessary. “I accept full responsibility” is corporate-speak empty hyperbole, and it has to end. Is he any more responsible than those who introduced the hole in the first place? No. Is he responsible for not adhering to agreements he was not privy to? No. He cannot “take full responsibility” because that is an utterly unreasonable burden to place upon him, a lone well-meaning actor who unbeknownst to him took a misstep. We need to discount this corporate-speak linguistic hyperbole.
Oh just imagine if politicians talked like that.. (off topic I know)
> At the time of posting, I didn't know about any embargo. My contact had told me there was no embargo, and her contact had told her the same thing.
I'm not saying it needs to be a public burning, but I sure hope someone figures out who dropped the critical metadata bit so they can avoid telling that party critical things in the future.
Eh. I'd rather we set up a system where we don't have to rely on people being 100 % perfect for it to work. I'd be much more interested in what went on simultaneously that caused the meta data to get dropped, rather than who happened to be the one to do it this one time.
Sometimes the "system" consists of identifying and removing or bypassing unreliable components.
If you could tell ahead of time which components would be unreliable, that would be a great idea. For the most part, we only discover the component was unreliable after it's caused trouble. Removing the component at that time gets you at best a very short-lived improvement to the system because it'll happen again and again with other components.
If the component removed is a human, the situation is worse. Then you're creating an incentive to hide mistakes and assign blame to others. This makes it even harder to improve the system properly, because then the information you need to improve the system no longer gets to you undistorted.
That's why we have to design systems that don't assume components are perfectly reliable, and ensure any human components that we're not out to get them, we're out to create an environment where they will find it hard to make mistakes.
As a general rule, systems that scapegoat people never fix the underlying root cause.
It was exploited publicly even before the wide publicity following your post so the ultimate negative impact that you may be imagining, is actually quite small. Once it is publicly exploited the benefit of coordinated disclosure delays is hard to square with the risk.
While it was exploited publicly, it wasn't until the HN post and especially a video was posted until the exploits exploded - don't underestimate how many eyes are on HN versus obscure darknet forums or private chat channels.
If someone (independently) found an exploit and weren't thinking of the public interest, they would want to keep it secret and for their own use.
The first rule of the fight club only applies if you know about the fight club.
If I get randomly punched by someone who was in the fight club, with no knowledge of their participation, am I now a member of the fight club.
I mean, if you want to be at that point
What is the fight club?
It's a quote from a 1999 film: https://en.wikipedia.org/wiki/Fight_Club
The second rule of fight club is DON'T TALK ABOUT FIGHT CLUB.
Not only that, it's also one of the very few instances in history where the film adaptation rivals the original work. But the book is great too; Palahniuk does amazing things with language.
We don't talk about The Fight Club.
the coordinated disclosure
what about the second rule of fight club?
You were not under any embargo. This only goes for the first person in the chain, whoever that was and there is absolutely nothing to apologize for.
As an Australian I thank you. I'd much rather have spent all day today patching things than tomorrow. It's Friday afternoon right now!
There's a rule about not pushing to prod on Friday. I feel the same should apply for security patches ;-)
I don’t know if embargoes have much effect anymore. They used AI to find this issue, meaning anyone else could also find the issue with time and the right model. The clock is always ticking after a CVE but now it ticks faster. I am certain that companies like Cellebrite are pointing a bunch of models at popular repos and filing away all the exploits they find in a database.
I also got that information before your post. I believe an embargo is impossible to keep up when it takes way too long to get a fix out.
"You never blow the cover of an agent in the field." --Jackson Lamb
Don't feel too bad. Just knowing a "10.0" RCE CVE is inbound w/o extra details doesn't get you very far. The actions of calif.io and Chainguard after your post was what caused the damage.
> Just knowing a "10.0" RCE CVE is inbound w/o extra details doesn't get you very far
Well that's exactly what chainguard claims though. "Find bad bugs in PHP" is the prompt they claim they used to rediscover the issue
You don’t have to wait for an HN post to run that, though
All vulnerabilities should be disclosed immediately, publically. That would over a short time lead to much better security practices.
Thanks Pete. I appreciate you taking the time to post here.
I really hope this isn’t chewing at you and that you’re just trying to do what you feel is the right thing by posting this.
The only bad guys here are those who exploit issues like this for unethical reasons.
You clearly aren't that person.
Regarding your perceived mistake, we all make them.
Please forgive yourself and understand that the negative feelings and thoughts you are having are a mark of decency but do not represent reality, and if they persist please seek help to get you back on track.
As an aside, to you and those like you who take on the task of fighting and protecting against these attacks with often nothing to gain except hurt like this:
Thank you for your service.
It's ok. Hopefully people took it seriously and updated quickly.
“I don't get it. Why are they confessing?” “They're not confessing.” “They're bragging.”
Hope you're doing alright mate. Seems like an honest attempt to do the right thing to me. Get some sleep.
Nah, you just repeated something that to you was a rumor.
There are companies whose "rogue" agents have hacked 6 7 companies and at least 2 3 govts. They haven't faced any consequences at all! They actually hacked into those companies
And there is you who is taking responsibility for an embargo you did not know about!!
> I take full responsibility for my actions.
Meaning? Like... global financial fallout?
> I have learned my lesson.
Meaning? Like... feel bad for a week, and then back to normal?
Who told you about it?
Related (where you already posted a bit of this):
Today's PHP security release, and how we're strengthening security
https://news.ycombinator.com/item?id=50182987
petecooper's post that apparently broke the embargo: https://news.ycombinator.com/item?id=50167754
yep, i saw it and ran the story on RuntimeWire once i confirmed with the watchtower X post.
and then everyone sht on me on reddit.
are you the guy working at Chainguard publishing images...?
Nope. No connection or affiliation with Chainguard at all:
https://news.ycombinator.com/item?id=50183326
Hah, I was trying to find this comment you made. There's confusingly also "Calif" in the mix (possibly closer to your string of people, but not your affiliation either). I wonder if, perhaps, it was easier to point to your HN post for bleepingcomputer, than what Chainguard did (you only expressed something was coming). It's all slightly immaterial since exploit attempts started 3-4 days.
No connection or affiliation to any of the organisations involved. I am a self-employed consultant for my day job, and a solo founder of a mental health non-profit on the side.
Honestly this has been the most fun week I've had, thanks to this leak. I'm grateful. Fun stuff happened, shitty companies learned the value of capable and fast engineering teams.
All in all, I think this leak increased the value of developers, so I am thankful for your leak.
hey Pete, don't carry the world upon your shoulders
This does feel like the fix took so long to bake and was so obvious way too many people who are into PHP knew about it, I knew about eMudra's cert issue as soon as Chrome and Firefox pushed updates to disallow them for instance because of reasons, and it wasn't even fixed after it was all made public on several platforms, I think Apple still hasn't fixed it properly, the trust Store is still fucked not sure if Safari was separately patched, maybe there is an os update in the last few hours I missed.
This entire RCE race is always annoying.
So much pearl clutching from these security companies. An embargo isn’t an enforceable contract, expect it to leak and be ready to fast forward your plan. Don’t start pointing fingers because people protected themselves or their customers once the dam broke.
You're a good person with good intentions. Hope you get some rest sir.
i forgive you
It was a good thing. It started useful conversations and taught valuable lessons that we probably wouldn't have faced with a more "traditional" vuln disclosure/patch process. We're going to see this a whole lot more with AI, probably as 0-days. It's better that we got the fire drill out of the way now.
OP is being a big man about this
Good shit
The security field is full of a bunch of self important jagoffs that think they are “super smart”. Its all a joke and its sad that anyone feels they need to play these stupid games.
Security is important but the people in the security field are clowns and it seems it has gotten worse over the last 20 years.
You broke nothing, it's just another day my guy. Go for a walk, drink some water, you know, take care of yourself - others failed to follow a process, not like you threw out POCs left and right.
Yeah, I actually first heard it here from you.
It's ok. It's just php.
It's okay. It's just 70% of websites.
So.. what has the effect of this all been? I understand that the early release of POC exploits increased the pressure for mitigations in advance of the patch release and caused a lot of work and stress. I have read the tweets from Cloudflare with statistics for malformed requests related to suspected exploit attempts.
I know breaches are often kept under wraps, and there's a long tail. But is there any evidence of impact at this point? Were any sites compromised that we know about? There was a lot of hullabaloo, with foreign and domestic LE agencies recommending sites be taken offline (holy shit!).
Was this like Y2K where, based on the coordinated action of many, a real issue was addressed in time? Or is it like Hale Bopp where everybody watched, and a few casualties in the preparation, but ultimately a nothingburger?
Flame on!
Can someone, with the benefit of hindsight, explain the embargo ? Why isn't there a clause "if the vuln leaks, embargo is broken" . I still don't understand the fuss.
Everyone knew about the vuln well before the patches going live. The embargo, at that point, made things worse.
In plain english, please.
The idea is that nobody has chance to have prior knowledge to identify or research the bug until it is generally possible to apply the fix immediately. If someone leaks the patch beforehand, others can inspect the bug and start exploiting it in the wild, while others have no chance to apply the patch.
Whether someone just leaked the CVE numbers or just ”there is a big bug in PHP” - that should not be enought to accelerate finding that bug. But other information that pins the bug for specific property or feature makes it easier to find and race against patching starts.
And at that point you don’t really know if someone really found it or not, since only few big organisations can collect that much data to detect successful exploiting. And most ”PoC”:s in the wild were credible-looking but vibecoded mess that did nothing what sources claimed.
Until there is absolute evidence that exploitation happens in the wild, succesfully, you don’t leak the patch or bug details beforehand.
In this case, the area where the bug is in PHP, leaked partially, but apparently there was no enough evidence that it was exploited succesfully or if any of the PoCs in the wild actually worked properly. So patch or related detailed information was not released before the initial schedule.
> And most ”PoC”:s in the wild were credible-looking but vibecoded mess that did nothing what sources claimed.
Could this be a variant of the survivorship bias? Maybe only the low quality PoCs ended up in the honeypots while the more capable actors avoided them.
Your later insights about the vuln POCs being a scam helped me better understand the situation.
I could see the value of the embargo in the old world of “6 uber security firms and a few bounty hunters” but the new world of pointing AI at the PHP source just seems flimsy.
It happens. Information cascades are messy, especially with multi-layered contacts
Thanks for owning it ,the best thing now is ensuring the patch details are clear once the official release drops. Moving forward is what matters
You’re setting a great example of how to be open and take responsibility, even when you’re not actually at fault. I really respect how you’ve handled this. You’ve also given the community a valuable reminder that some security disclosures may be under embargo, and that it’s worth having in mind before sharing.
> I take full responsibility for my actions.
I would like this to materialize, but that's just empty words.
> It was not remotely my intention to cause any of the subsequent side effects.
What were your intentions, then?
> my intention was to alert others in a similar position to be aware of the importance/urgency of the updates.
Have you lived under the stone for the past few years with respect to cybersecurity? Yes you have. All of you ignore security as in "it doesn't apply to me", "they won't break into my software because I'm truly the best 10x developer in the world" etc.
> I have learned my lesson.
I certainly doubt that. A person with IQ only a tiny bit bigger than Trump's wouldn't seek public attention so badly to disclose a security hole in a public forum.
When such praise-seeking habits have stopped being embarrassing?
x
The modern version:
You have sinned.
Your penance:
Three Hail Claude's, one Our GPT and an act of loop engineering.
Two minutes of silent prompting.
and 1 hour, writing code, no AI.
That's a bit harsh.
What joy. Not much of a punishment.
In the name of The Large Language Model, The Harness and The Tokens!
Amen
Really hope you are not in the US. This kind of admission would be used against you in a civil trial, and lawyers love to sue for just about any reason under the sun. Someone could drag you to court and you might be able to defend yourself, but at an extremely painful cost both financially and emotionally. While I think that would be utter bullshit, that does seem to be the way this country operates now.
That should be no problem: he posted exactly *because* he meant to take full responsibility for his actions.
I am not in the US.
The parent is getting downvoted but he is absolutely right. Do not ever do what you just did. It makes you an easy target for a frivolous lawsuit in the US or in other countries because of the potential liabilities in your message, they are real. Even if you didn't do anything wrong, you are painting a target on your back, for no reason. I would ask HN to delete that thread directly.
You cannot foresee the financial/civil consequences of that RCE and some people might be looking for scapegoats in the future.
Anyone can sue anyone else for any reason. We should encourage honesty and transparency, not discourage it.
While possibly true in theory, in practice it's not. Courts and judges do not take kindly to frivolous or unsubstantiated claims, and any halfway good attorney would advise you to not waste your money and your time on a claim you are unlikely to win or be able to collect on.
People and companies do get sued with the full intention of a settlement and it never going to trial. But that only happens when the defendant has deep pockets or good insurance.
This isn't a matter of transparency, was the OP legally bound not to disclose the RCE? That whole "embargo" was pure courtesy between the people involved. The OP committed no fault at first place and the exploit was already in the wild anyway. Whoever really wanted to exploit it before the patch could without using the OP as a source.
> Anyone can sue anyone else for any reason.
And it's easier to have cause to sue when someone publish an "admission of guilt", which makes a lawsuit harder to dismiss thus costlier.
To win, the cause needs to be a broken law or contract. Neither are here.
There's at least some evidence in the other direction.
https://med.stanford.edu/news/all-news/2017/10/in-patient-in...
> New research shows that discussing hospital errors with patients leads to better patient safety without spurring a barrage of malpractice claims.
It has nothing to do with anything I said.
Sure it does. Saying sorry sometimes reduces liability risk, which is quite counterintuitive at first.
> Sure it does. Saying sorry sometimes reduces liability risk, which is quite counterintuitive at first.
No it doesn't have anything to do with what I said. The OP has no contractual or legal obligations toward any community, unlike a doctor and their patients in case of medical error.
edit:
> A doctor has no contractual obligation to apologize, either.
> But it can help prevent things from going to lawsuits, versus the "fuck you I admit nothing" approach that sometimes just pisses people off.
OP isn't a doctor as far as you know, and isn't apologizing to a specific person. OP can't be bound to an embargo he didn't know about and wasn't involved in any decisions.
You're completely off topic and you just want the last word.
Your ramblings have nothing to do with anything I have said.
A doctor has no contractual obligation to apologize, either.
But it can help prevent things from going to lawsuits, versus the "fuck you I admit nothing" approach that sometimes just pisses people off.
So who will be sued and/or state prosecuted?
This is definitionally humble bragging. First of all the apology is in bad faith because there is nothing to apologize for. We are all giving the discovered CVEs attention (the brag component) under the guise of an apology (the humble component). I am being downvoted by well meaning, people pleasers that don't mind if they waste our time and attention doing it. Sounds great.
Is there any insight on if this was a "Bug" or a "Sophisticated Feature" put there by state actors long ago that was now just discovered.