I really wish the analysis didn't cut off at exactly 10 numbers. Rerunning the query with higher LIMIT, I noticed the following:
- The very next two numbers are 100 and 0. While 100 is consistent with the article's explanations, 0 still placing high despite fixing the query warrants further investigation. A quick glance at returned headlines shows the problem of phantom zeroes is not, in fact, fixed.
- The query doesn't group decimals and integers together. 2.0 is at #17 with 10k hits, while 1.0 is at #26 with 5k hits. So not only is the "version number" explanation for top numbers wrong - the claim that earlier version numbers are more common than later version numbers is wrong too.
The article mentions one reason for small numbers predominating:
> The first is listicles. “3 ways to do X”, “5 things I wish I’d known”, “10 tools I actually use”. The listicle is practically the native unit of the link post, and its count is almost always small and round. Nobody writes “17 things I wish I’d known” unless they genuinely have 17 things, which is rare, whereas “5 things” is a decision made in a title editor.
But doesn't the title mangler specifically try to strip out numbers that look like they're part of listicle titles?
I once did some small work with a mathematician who was an online friend of mine. He was really into Benford's Law. A lot of mathematicians seem to love Benford's Law, and I wonder why that is. I'm not sure if I was rude to him, but we lost touch, and I miss him. I learned so much from his code.
Probably because it's so useful in detecting the "smell" of fraudulent numbers. Fraud is, of course, going to be most common where money is involved, but it can show up in many other places: for example, someone might be tempted to fudge their scientific data if their career depends on getting numbers that aren't quite where they want the numbers to be...
If a number distribution that should be following Benford's Law isn't, that's nowhere close to actual proof of fraud, but it can be a pointer helping you know where to start looking. Because if all the files except this one have numbers that follow Benford's Law, yet this one doesn't, then that's the place where you start digging for other, more positive, evidence of manipulation.
I really wish the analysis didn't cut off at exactly 10 numbers. Rerunning the query with higher LIMIT, I noticed the following:
- The very next two numbers are 100 and 0. While 100 is consistent with the article's explanations, 0 still placing high despite fixing the query warrants further investigation. A quick glance at returned headlines shows the problem of phantom zeroes is not, in fact, fixed.
- The query doesn't group decimals and integers together. 2.0 is at #17 with 10k hits, while 1.0 is at #26 with 5k hits. So not only is the "version number" explanation for top numbers wrong - the claim that earlier version numbers are more common than later version numbers is wrong too.
Thanks for providing the query and a link to execute it (I didn't know about that one). Now I can play with it and find the most popular words...
"to", "the", "of". I don't know what I expected.
For determining words that are important doing something like a tf-idf would make more sense, or doing some key word analysis like RAKE or YAKE.
https://en.wikipedia.org/wiki/Benford%27s_law
Yes, as linked in the post
I imagine "one" is usually spelled out in titles.
Until you replace it with "1" because your title is too long for HN to accept as is.
The article mentions one reason for small numbers predominating:
> The first is listicles. “3 ways to do X”, “5 things I wish I’d known”, “10 tools I actually use”. The listicle is practically the native unit of the link post, and its count is almost always small and round. Nobody writes “17 things I wish I’d known” unless they genuinely have 17 things, which is rare, whereas “5 things” is a decision made in a title editor.
But doesn't the title mangler specifically try to strip out numbers that look like they're part of listicle titles?
Probably 42, but I could be mistaken...
(2026)
That would be this year, and the story was posted ... today.
HN's convention is that the year is added for stories over one year old to indicate non-current (though still possibly interesting) submissions:
<https://news.ycombinator.com/item?id=39958349> <https://news.ycombinator.com/item?id=37728779>
My guess is 1 (or 1.0) declaring something has reached 'stable'
Edit. Perhaps the title should be 'parsing numbers is hard'.
I once did some small work with a mathematician who was an online friend of mine. He was really into Benford's Law. A lot of mathematicians seem to love Benford's Law, and I wonder why that is. I'm not sure if I was rude to him, but we lost touch, and I miss him. I learned so much from his code.
Probably because it's so useful in detecting the "smell" of fraudulent numbers. Fraud is, of course, going to be most common where money is involved, but it can show up in many other places: for example, someone might be tempted to fudge their scientific data if their career depends on getting numbers that aren't quite where they want the numbers to be...
If a number distribution that should be following Benford's Law isn't, that's nowhere close to actual proof of fraud, but it can be a pointer helping you know where to start looking. Because if all the files except this one have numbers that follow Benford's Law, yet this one doesn't, then that's the place where you start digging for other, more positive, evidence of manipulation.
Thank you!
We're number 1!
benfords law strikes again. useful for fraud analysis too
You missed a great opportunity to title this "The 10 most popular numbers in Hacker News titles"!
You'll never guess number 6! Seriously, why would you think that 6 is one of them, there are so many better numbers out there
Now that's Numberwang!