Noting this article skips over any discussion about why, or potentially (in the eyes of the law at least) legitimate reason an ISP might do this.
Many ISPs in many countries are forced to do this for certain enforcement action that gets put upon them by their respective states for things such as age gating, censorship, monitoring, it varies state by state.
And to be clear this is in no way an endorsement of the practice, and I am confident no ISP WANTS to do this.
I am speaking here as the owner of a UK ISP (who does not currently do this).
Last I checked Verizon FIOS still has the opt-out. You have to go into your account preferences and manually disable it because spying is on by default.
This is a good reminder to login and take a look, I wouldn't put it past them to change the setting name and reset the default.
edit: apparently Verizon has zero record of the username I've used for my FIOS account for years (and have saved in my password manager). Their account reset also fails with an error. What a dumpster fire of a company.
Yep, I've been in Verizon online account purgatory for awhile now. They merged my mom's with mine and nothing works to reset or restore online account access for even bill pay. I have to push their payment out via billpay. I really need to move that phone number over to voip, I kept the classic POTS line as a type 1 diabetic I like that it stayed on during both NYC blackouts in 2003 and 2012. But now I'm in a fiber only building and that redundancy is mostly moot.
I don't use their resolver at all. Their opt-out was more of an all-encompassing "let us take whatever data we can manage to get" than DNS specifically.
There were certainly a number of ISPs in the UK that tested ways to inject ads in content in the 2000s, there was quite a hoo-hah about it in some forums. That was usually by modifying HTML content in transit via transparent proxies though, not DNS poisoning, but it shows that the intent exist{ed|s}. This is one of the reasons that HTTPS-everywhere is a good idea even if you aren't concerned about snooping in general.
AT&T still does this, if you're using their (intentionally terrible) standard gateway device the default DNS will return a "search page" instead of NXDOMAIN which predictably has all sorts of negative impacts on non-browser devices but the wastes of oxygen responsible for this don't care as long as they see that delicious ad revenue.
Some mobile phone network carriers would inject certain PII (such as your mobile number) into HTTP request headers sent to ad networks. I'm sure they still do it, but by requiring the site to have some extra request to be made instea.
Some carriers also passed info in these headers to be used to make billing calls to the Telco, such as subscribing to premium services with minimal (if any) consent from the user.
I got stung by this years ago. An ad for mobile ringtones showed up on some news site I visited on my phone, I scrolled past without clicking. Next minute I get a text telling me I've subscribed signed up to a (iirc) $25/month ringtone plan and that it'd show up on my phone bill. It took an hour on the phone to my carrier to block the charges and get a refund. The ringtone company kept messaging me for a year that I needed to contact them and "fix" the issue.
As someone who was sometimes using http as a back end transport for "edi" data (hey at the time it was usually ftp in that industry so no worse, but at least http got around nat problems than even pasv didn't), one time in the mid 00's I was in a coffee shop and just speechless flabbergasted outraged to discover the contents of a random web page I happened to know the contents of get some extra html and js injected into it from the isp (the isp not the coffee shop).
"WHO THE F gave them the right to modify the data in my tcp session!!!!".
Spying on it I assumed but modifying one byte of it? A naively written edi that did not expect that would be completely destroyed.
Probably shouldn't have been so stupid for 15 years already by that point. There is no such thing as the right, there is only the ability.
If someone has the ability to do something, they will. Reputation, right & wrong, none of that actually exists or matters.
I didn't track down if the mechanism was based on dns. Could have been but I don't see why it would have had to be. But I was also not attempting to use any non default dns on that device at that time anyway. Obviously since they are trasporting unencrypted data they could modify it any time simply because it's in their hands.
dns would just be one handy low-hanging-fruit way to detour the data to a convenient dedicated purpose server to do the work. But not special or necessary at all. If they want to do the same thing without hijacking dns, they can with almost not even any difference in total effort.
IIRC, early into ATT’s US fiber / gigabit rollout, they wanted an additional $40 / month to forgo collecting and selling your browsing history (DNS queries).
If you're a UK ISP owner, you're almost certainly a RIPE NCC member, given the UK is within its regional scope. Therefore, you've probably spoken to other RIPE NCC members via mailing lists or meetings at least once. Since I also represent a provider that is a RIPE NCC member, I can object that not everybody shares the same ideology you noted. There are some providers that actually speak in favor of such over-regulation and censorship. Everybody can read and/or participate in the RIPE Discussion List, though, and you will be amazed by some responses from ISPs that speak in favor of such measures.
But the good news is that at least the majority (for now) is against it.
I speak for another RIPE NCC member when I say that that ideology is limited to a very small minority.
Even outside ideology, plain business sense makes ISPs not want to deal with censorship & co. It's work to do that, hence costs money. But there are also ISPs resisting out of ideology and civic duty.
One massive reason is they redirect requests for videos to closer (or local) caches of videos for netflix (directly from youtube placing it with a large enough ISP), or local caching at the ISP level so less traffic hits the internet.
If you've ever been hit with a "webpage isn't updating" issue when someone else can see it, often it's different DNS servers.
It's best to never use the ISP's DNS servers, regardless of the reason why they might do what they do.
this is valid, in NL most DNS is also heavy monitored and intercepted to do blocks if mandated etc. if you want to get around it u can tunnel ur tlsdns thingy to another place and they essentially cannot intercept it, they can still block the IPs it resolves to so its hit and miss depending on how the other end is implemented.
its usually not nefarious, that is just the simpelest (and usually, not always) wrong explanation.
people who think this is not normal probarbly didnt experience what kind of shit u could find on the pre google pre monitored internet. for sure shit u do not want your kids to find. if u dont understand, then likely u dont have kids and very weird fetishes...
Why do you assume the others have same ethical compass as you? I hope everyone has the same compass but we have seen again and again trust being broken.
ISPs talk to each other, and people move jobs between them. It's not a uniform community but you certainly get a sense of the general spread of opinions after a while.
There are of course outliers but most people I know in the field care as much about personal rights & freedom as anyone else. Maybe even a small bit more.
My workplace is messing with DNS as well, probably to filter specific websites, and I would not mind it since for the context of my usage that is perfectly fair.
But their DNS responses have latency spikes every couple of minutes of 1-2 seconds that was driving me crazy. I simply enabled DNS over HTTPS in Firefox and everything feels so smooth now.
I have no idea how my coworkers seem to be so insensitive to it.
If you're not working on something highly secretive and valuable (with a salary to match) then they're doing you a favor by flipping out over something so stupid.
They might be proxying all your traffic through a "WAF". I've noticed that even quite a lot of regular websites linked on HN on "non standard" domains are simply unresolvable on my work network.
Does anyone know if T-Mobile does this too? I'm considering moving fiber to them instead of AT&T, but have stayed mostly due to "better the devil you know..."
Things are crazy now. Our TVs snapshot the stuff we watch; our telephones, watches, and TVs are constantly listening what we say; our PCs have undocumented "remote administration" devices that we can't opt-out; every internet service out there is stealing every data that it physically can and selling it to a long list of organizations, including some military ones...
That is regulation! You do actually have to win the argument for liberalism in public if you don't want your fellow citizens to keep trying to stop you doing this.
Telecom Italia locks my DNS server (I cannot change it).
I ended up using PIHole as DHCP, to drive home devices to PIHole, and control my privacy.
The trick is simple: disable DHCP on your router, and activate it on PIHole.
As a bonus, PIHole on my RasperryPI2 is way faster than the little chip on my provider Router, and can be patched automatically with unattended upgrades.
And if you can't disable DHCP or set the router in bridge mode, you can just add another router (that you control) between your ISP device and the rest of your network. I've been running this setup for over a decade and didn't run into any noteworthy issues. Leave the ISP-infested router outside your network.
Note that there are ISPs that actively hijack traffic to UDP port 53 further inside their network and transparently maps that onto their internal DNS servers.
Recently went down a little rabbit hole to get my house in order when I noticed weird things showing up at https://dnscheck.tools/ . (I don't really understand how these checks work though)
For various reason, I also setup an Unbound who proxy to quad9 and is only accessible through DNS over HTTPS/QUIC and is behind a reverse proxy with auth (because being an open dns resolver on the internet is still a bad idea). This is nice because I can then have a dnsproxy or similar on my various devices, who all point to it and be pretty sure nothing can hijack my DNS queries. I also come with the benefit of forcing thing like a minimum TTL to avoid dumb services who decide to put a 10 seconds TTL to all their DNS entry...
The point of running a local unbound is indeed to turn on DNSsec validation. After that, it's the domains' fault for not using DNSsec. But at least your own stuff won't be tampered with, because you use DNSsec on your own domains, right? Right?
:-(
Coincidentally, a bunch of free wifi stuff blocks the root key queries unbound does at startup, resulting in nothing working until you get those records in there somehow (e.g. load_cache).
DNSSEC as designed is pretty useless. It was designed in an era where computers could not be relied upon to do cryptographic validation of something like DNS records, so that duty was delegated to DNS servers like the ones ISPs ran, where records could be cached.
There are also potential downsides (like someone being able to enumerate your subdomains, if you care about such a thing) and it's often switched off by default on operating systems consumers use.
However, it's the only thing that validates DNS records at this moment. That alone makes it worth enabling, in my opinion. For many domain owners, you only need to tick the checkbox in your registrar's domain control pane. If you use a modern DNS server, it's also one config flag and maybe copy/pasting a handful of fields away.
I'll gladly turn off DNSSEC the moment we come up with something better. DoT/DoH doesn't solve the same problem and no real solutions seem to have been proposed. That said, modernisation efforts are still being done. Smaller keys and modern cryptography rather than the massive RSA ones for instance.
I can't speak about the strength or weaknesses of the whole concept.
Out of curiosity I enabled DNSSEC checking beginning this year. I mostly noted nothing in my browsing. However, with 2 services I noted obscure problems that worked only partially. Some Jira plugins and Swedish railway SJ. I needed to excempt some of their domains from being checked to use their services. Of course you waste a couple of hours every time it happens because you don't think of DNSEC first when some web-based services behaves weirdly. There might be more requests not served to my browser, but I just never noticed or bothered to debug.
So not fit for general use. And even as an experienced developer it's not easy to understand why an exception would be needed. A bug in systemd? Incorrect data in DNS? A real attack?
Edit: I don't use standard DNS on port 53 at all. Blocked outgoing requests on that port. Everything DoH or DoT. Of course I need to trust those endpoints at least in repect to confidentiality. And most domains are unsigned. At least for those unsigned ones I need to just trust also in respect to integrity.
DNSSEC failures need to be louder. If websites that fucked up their DNSSEC would get the same error messages as websites that fucked up other parts of their security, people wouldn't be so eager to turn off security festures as workarounds.
Give SJ the option to turn off DNSSEC and their website will stay broken. Give people a "someone might be tampering with your connection" popup and they would start caring.
There are also other issues with DNSSEC that need working out, but I still believe it does much more good than it does bad. Leaving everything in the background with an easy-to-disable setting makes it next to useless.
Full ACK. As a part of this, DNSsec validation needs to be as close to the client app as possible and reasonable; definitely never off-host. (Caches can do additional validation, but that makes the error reporting complicated.)
corollary: the AD bit should never be relied on on anything actually on the network. Maybe on localhost, but even there it's questionable in UX results.
...99% of the internet runs on DV certificates; that's what ACME & Let's Encrypt are at the end of the day.
(You might also note I didn't actually speak in favor of DNSsec, I'm just arguing that running unbound locally without having DNSsec validation enabled isn't particularly useful. I am, generally, in favor of rolling out DNSsec, but it certainly has its problems. It doesn't help against ISP tampering on unsecured domains, for example. DoT and DoH are better against that, if [and only if] you have a trustworthy cache to point them at.)
Adoption at the TLD is high as it is expected of the TLD operators but the second level is very low. So .org itself is signed but .org second level domains are just below 5% adoption overall. .com second level is around 4.2%. .net second level are a little higher at around 5.2%. So most people's answer would be a solid no. HN audience may be some of that 5%.
DNSSEC adoption rate means that this would be insufficient to detect (let alone prevent) that type of tampering for most domains. The correct solution is identified the the first paragraph of the post, which is to use a tamper resistant transport to the dns server (eg DoT, or DoH if you like everything being wrapped in http)
There's a difference between what you're likely referring to, where the ISP's own DNS, which most people are using by default, returning particular results, and what the author is showing here where the ISP is actively tampering with outside DNS responses to insert incorrect responses.
Until now, I was not aware of any ISP doing the latter.
Only one ISP I've used ever did this (VodafoneZiggo) and that was over a decade ago, and only to redirect domains that couldn't be found to ad-filled "search" pages.
It's not universal, the occurance or this type of abuse seems to vary wildly from country to country
Illegal, where exactly? In various western countries ISPs are by law required to block some websites. In other countries, ISPs ar eby law required to have escape hatches to be able to shut down the internet. In yet others, everything runs behind a firewall. Take your pick, I guess?
Some countries have laws around Net Neutrality or Open Internet that can protect you from this, but even those countries have other legislations that can superseed that in specific cases. (e.g. hijacking DNS can be completely legal for some child protection laws, or to uphold laws that makes some web sites illegal)
I’m using NextDNS over https and frequently experience connection freezes with both Spectrum and T-Mobile. I don’t know what they’re trying to do, but I’m not happy with the way they’re trying to block basic services.
Unlike OP, I have all my DNS route completely to my home DNS server, and only make changes to that container and my VM host during a "change window" meaning not during work hours. I block port 53 outbound entirely and have host entries for the two domains that my DNS server uses for DoH. If I had a better firewall on the edge, I would leave 53 open for my DNS container, but this workaround seems suitable enough for the home network.
Reminds me of the Xfinity "Your personalized WiFi summary" emails. Despite returning their router, using a third party almost-layer-2 modem, my own openwrt router (with its own wifi, thank you very much), their emails seem spooky.
They have a counter of all of my devices, somehow (i mean... i guess their count is wrong, but it's infuriating that they try to keep track).
All of that could be done without "ISP was hijacking my DNS". DPI/SNI sniffing is more accurate and allows you to get more detailed statistics (eg. "20% of your bandwidth goes to youtube"
No. That's not within the design function of the roots.
It's very much part of the design that roots and caches are separated. The security properties necessary between root and cache (authenticity) are fulfilled by DNSsec, not DoT/DoH (primarily privacy). And most TLDs are DNSsec signed, so this part is in fact OK. (Both give some level of tamper protection, in a different manner though.)
You're probably saying this because you run a local cache directly against the roots. If you need DoT/DoH, you shouldn't be doing that and rather forward your queries to a cache somewhere else.
(To make sure there's no misunderstanding here: DoT and DoH do absolutely nothing to guarantee the data you're getting is what the roots/TLD/owner published. It only ensures you're talking to the cache you think you're talking to. The cache can serve to you whatever it wants.)
Switching too is funny. You are at the mercy of what isp
ran a line to you. Starlink will fully cooperate with government too. They are a military contractor after all and also at the mercy of musk deciding dumb stuff.
A VPN just moves the attack vector from the ISP to the VPN provider, it doesn't protect you from this in any way at all. Better to use an ISP you trust, though that's difficult to find.
I don't trust any public VPN provider because they all start off by lying about what their product does.
What about it? Unless they've hijacked your local cert store somehow (which corporate machines do), they can't see inside the TLS. They can detect that you're connecting to somewhere and probably infer that it's some VPN, but only the most repressive countries actually block VPNs altogether.
Isn’t that the same as no vpn with extra steps? You are tunneling traffic from a computer you own to a computer you also still own. Third party vpn the whole point is I don’t run it.
If you set up a VPN endpoint on a Hetzner box, you're trusting Hetzner not to MITM your DNS queries. They're likely more trustworthy than your local ISP (although that leaves you more identifiable to folks running the servers or snooping on server-side traffic, and Cloudflare will hate you).
That threat model tradeoff is good for many people.
Not if they are only using it to route the DNS traffic (though in that case a cheaper VPS might be better), which will work if the ISP is only poisoning DNS replies and not (which is often the case) blocking traffic to the target servers entirely.
The problem with this is you are adding a moving part to your DNS lookups. It might be worth adding two on separate VPS hosts and have your resolver fall back to the second if the first goes down.
Another, usually more minor, problem is that some services (like CDNs) use the source of your DNS query to modify the response so you get sent to a topologically near-by end-point. This means you might notice such services being slower as you are sent to a PoP near your VPN host not one closer to your actual location.
Oh well OP was doing shady stuff ... then yeah they should continue with using VPN provider. VPN providers are shady AF, I don't think there is VPN provider I would trust more than my local internet provider.
Well that wouldn’t be running your own vpn. Hetzner is. They’d let you do this? Seedbox is against their tos hard to imagine they’d be a good choice to run a vpn for your linux isos.
Can't speak for Hetzner, but considering VPN use in Russia and China - I'd imagine most EU\US vps providers have someone's 3x-ui, remnawave or naked v2ray installed.
Plus - how are they going to detect your remnawave node sitting on 443 port? For them it's just some traffict.
And if you are worried about the amount (torrents) - you don't really need to torrent though them. You just need an access to a torrect tracker to get your magnet link and meta info.
Most default configs for 'ray have torrents locks for this very reason - you communicate with other people (seeds\peers) directly.
Noting this article skips over any discussion about why, or potentially (in the eyes of the law at least) legitimate reason an ISP might do this.
Many ISPs in many countries are forced to do this for certain enforcement action that gets put upon them by their respective states for things such as age gating, censorship, monitoring, it varies state by state.
And to be clear this is in no way an endorsement of the practice, and I am confident no ISP WANTS to do this.
I am speaking here as the owner of a UK ISP (who does not currently do this).
I'm pretty confident ISPs do want hijack DNS since they've been doing it for decades in order to push ads to their customers.
Examples?
Comcast and/or verizon did it in the 2010s, there was an opt-out in the account dashboard.
It's sort of an absurd question, the prevalence of such things by ISPs was the literal impetus for DoH and DoT.
Last I checked Verizon FIOS still has the opt-out. You have to go into your account preferences and manually disable it because spying is on by default.
This is a good reminder to login and take a look, I wouldn't put it past them to change the setting name and reset the default.
edit: apparently Verizon has zero record of the username I've used for my FIOS account for years (and have saved in my password manager). Their account reset also fails with an error. What a dumpster fire of a company.
Yep, I've been in Verizon online account purgatory for awhile now. They merged my mom's with mine and nothing works to reset or restore online account access for even bill pay. I have to push their payment out via billpay. I really need to move that phone number over to voip, I kept the classic POTS line as a type 1 diabetic I like that it stayed on during both NYC blackouts in 2003 and 2012. But now I'm in a fiber only building and that redundancy is mostly moot.
You should be NATing outbound 53 to a local resolver that upstreams to DoH/DoT. Relying on such an opt-out is rather optimistic.
I don't use their resolver at all. Their opt-out was more of an all-encompassing "let us take whatever data we can manage to get" than DNS specifically.
There were certainly a number of ISPs in the UK that tested ways to inject ads in content in the 2000s, there was quite a hoo-hah about it in some forums. That was usually by modifying HTML content in transit via transparent proxies though, not DNS poisoning, but it shows that the intent exist{ed|s}. This is one of the reasons that HTTPS-everywhere is a good idea even if you aren't concerned about snooping in general.
An example I remember is when Virgin Media replaced any DNS queries returning NXDOMAIN with their own "search page" plastered with ads.
https://shkspr.mobi/blog/2021/03/advanced-network-error-sear...
AT&T still does this, if you're using their (intentionally terrible) standard gateway device the default DNS will return a "search page" instead of NXDOMAIN which predictably has all sorts of negative impacts on non-browser devices but the wastes of oxygen responsible for this don't care as long as they see that delicious ad revenue.
Verisign tried this trick too.
https://dn.org/the-sitefinder-debacle-that-shook-the-interne...
Ah, yes, I forgot the NXDOMAIN hijacking.
I was thinking about interfering with existing sites.
Some mobile phone network carriers would inject certain PII (such as your mobile number) into HTTP request headers sent to ad networks. I'm sure they still do it, but by requiring the site to have some extra request to be made instea.
Some carriers also passed info in these headers to be used to make billing calls to the Telco, such as subscribing to premium services with minimal (if any) consent from the user.
I got stung by this years ago. An ad for mobile ringtones showed up on some news site I visited on my phone, I scrolled past without clicking. Next minute I get a text telling me I've subscribed signed up to a (iirc) $25/month ringtone plan and that it'd show up on my phone bill. It took an hour on the phone to my carrier to block the charges and get a refund. The ringtone company kept messaging me for a year that I needed to contact them and "fix" the issue.
Insanity that people used to pay more for goofy little midi files than they do now for Netflix.
As someone who was sometimes using http as a back end transport for "edi" data (hey at the time it was usually ftp in that industry so no worse, but at least http got around nat problems than even pasv didn't), one time in the mid 00's I was in a coffee shop and just speechless flabbergasted outraged to discover the contents of a random web page I happened to know the contents of get some extra html and js injected into it from the isp (the isp not the coffee shop).
"WHO THE F gave them the right to modify the data in my tcp session!!!!".
Spying on it I assumed but modifying one byte of it? A naively written edi that did not expect that would be completely destroyed.
Probably shouldn't have been so stupid for 15 years already by that point. There is no such thing as the right, there is only the ability.
If someone has the ability to do something, they will. Reputation, right & wrong, none of that actually exists or matters.
I didn't track down if the mechanism was based on dns. Could have been but I don't see why it would have had to be. But I was also not attempting to use any non default dns on that device at that time anyway. Obviously since they are trasporting unencrypted data they could modify it any time simply because it's in their hands.
dns would just be one handy low-hanging-fruit way to detour the data to a convenient dedicated purpose server to do the work. But not special or necessary at all. If they want to do the same thing without hijacking dns, they can with almost not even any difference in total effort.
IIRC, early into ATT’s US fiber / gigabit rollout, they wanted an additional $40 / month to forgo collecting and selling your browsing history (DNS queries).
Time Warner Cable used to do it. I haven't checked since they became Spectrum.
Deutsche Telekom (largest ISP in Germany) did this, not sure if they still do or were ordered to stop by court yet
If you're a UK ISP owner, you're almost certainly a RIPE NCC member, given the UK is within its regional scope. Therefore, you've probably spoken to other RIPE NCC members via mailing lists or meetings at least once. Since I also represent a provider that is a RIPE NCC member, I can object that not everybody shares the same ideology you noted. There are some providers that actually speak in favor of such over-regulation and censorship. Everybody can read and/or participate in the RIPE Discussion List, though, and you will be amazed by some responses from ISPs that speak in favor of such measures.
But the good news is that at least the majority (for now) is against it.
I speak for another RIPE NCC member when I say that that ideology is limited to a very small minority.
Even outside ideology, plain business sense makes ISPs not want to deal with censorship & co. It's work to do that, hence costs money. But there are also ISPs resisting out of ideology and civic duty.
One massive reason is they redirect requests for videos to closer (or local) caches of videos for netflix (directly from youtube placing it with a large enough ISP), or local caching at the ISP level so less traffic hits the internet.
If you've ever been hit with a "webpage isn't updating" issue when someone else can see it, often it's different DNS servers.
It's best to never use the ISP's DNS servers, regardless of the reason why they might do what they do.
> It's best to never use the ISP's DNS servers, regardless of the reason why they might do what they do.
But providers overwriting DNS packets regardless – this is the topic of the article :)
P.S.: And yes, there are ways around this like DNSsec etc.
this is valid, in NL most DNS is also heavy monitored and intercepted to do blocks if mandated etc. if you want to get around it u can tunnel ur tlsdns thingy to another place and they essentially cannot intercept it, they can still block the IPs it resolves to so its hit and miss depending on how the other end is implemented.
its usually not nefarious, that is just the simpelest (and usually, not always) wrong explanation.
people who think this is not normal probarbly didnt experience what kind of shit u could find on the pre google pre monitored internet. for sure shit u do not want your kids to find. if u dont understand, then likely u dont have kids and very weird fetishes...
Why do you assume the others have same ethical compass as you? I hope everyone has the same compass but we have seen again and again trust being broken.
ISPs talk to each other, and people move jobs between them. It's not a uniform community but you certainly get a sense of the general spread of opinions after a while.
There are of course outliers but most people I know in the field care as much about personal rights & freedom as anyone else. Maybe even a small bit more.
This is a complete non sequitur
My workplace is messing with DNS as well, probably to filter specific websites, and I would not mind it since for the context of my usage that is perfectly fair.
But their DNS responses have latency spikes every couple of minutes of 1-2 seconds that was driving me crazy. I simply enabled DNS over HTTPS in Firefox and everything feels so smooth now.
I have no idea how my coworkers seem to be so insensitive to it.
Enabling DoH on a locked down workplace network sounds like a way to get fired.
But I guess that if they let you do it they're probably not competent enough to catch you.
No way you would get fired for this, come on man. Maybe someone from IT would ping you to turn it off, worst case.
Depends on the workplace, but I've definitely seen people fired for less.
If you're not working on something highly secretive and valuable (with a salary to match) then they're doing you a favor by flipping out over something so stupid.
It’s a university network so things are not as strict as other work places.
They also do not pay for my device, so it’s only fair I can configure things my own way.
I’m pretty sure the worst that could happen is they would ask me to turn it off.
Have you bothered asking your IT to look into it?
They might be proxying all your traffic through a "WAF". I've noticed that even quite a lot of regular websites linked on HN on "non standard" domains are simply unresolvable on my work network.
AT&T in the US does this. Traditionally they did this so that failed lookups were routed to their search engine, which of course fed users ads.
At least they no longer insert ads straight into web pages. Thank goodness for https.
Does anyone know if T-Mobile does this too? I'm considering moving fiber to them instead of AT&T, but have stayed mostly due to "better the devil you know..."
I fear a future where pretty much everything is trying to hack you or doing something which is indistinguishable from the meddling of a hostile actor.
I suppose the answer is politics rather than technology: Get enough people together to make a rule against it, and at least it will become niche.
Hello from 9 hours in the future.
Things are crazy now. Our TVs snapshot the stuff we watch; our telephones, watches, and TVs are constantly listening what we say; our PCs have undocumented "remote administration" devices that we can't opt-out; every internet service out there is stealing every data that it physically can and selling it to a long list of organizations, including some military ones...
I'm not sure regulation alone would solve it. Depending on the country, governments themselves sometimes require ISPs to interfere with DNS
That is regulation! You do actually have to win the argument for liberalism in public if you don't want your fellow citizens to keep trying to stop you doing this.
Telecom Italia locks my DNS server (I cannot change it).
I ended up using PIHole as DHCP, to drive home devices to PIHole, and control my privacy.
The trick is simple: disable DHCP on your router, and activate it on PIHole. As a bonus, PIHole on my RasperryPI2 is way faster than the little chip on my provider Router, and can be patched automatically with unattended upgrades.
And if you can't disable DHCP or set the router in bridge mode, you can just add another router (that you control) between your ISP device and the rest of your network. I've been running this setup for over a decade and didn't run into any noteworthy issues. Leave the ISP-infested router outside your network.
Note that there are ISPs that actively hijack traffic to UDP port 53 further inside their network and transparently maps that onto their internal DNS servers.
This is why you would want to setup DNS-over-TLS or DNS-over-HTTPS to, say, 1.1.1.1. They can’t hijack DNS traffic that looks like HTTPS traffic.
Pretty sure that you can use DHCP and still override the DNS servers with most DHCP clients.
Yes but you need to do it MANUALLY on every client. Using the PIHOLE DHCP automate this and simplify the setup.
I think they're saying that their ISP's router does not allow this, thus the need in their case to use third-party DHCP.
Recently went down a little rabbit hole to get my house in order when I noticed weird things showing up at https://dnscheck.tools/ . (I don't really understand how these checks work though)
For various reason, I also setup an Unbound who proxy to quad9 and is only accessible through DNS over HTTPS/QUIC and is behind a reverse proxy with auth (because being an open dns resolver on the internet is still a bad idea). This is nice because I can then have a dnsproxy or similar on my various devices, who all point to it and be pretty sure nothing can hijack my DNS queries. I also come with the benefit of forcing thing like a minimum TTL to avoid dumb services who decide to put a 10 seconds TTL to all their DNS entry...
Hardly a new thing.
Oh and btw, it's not "DNS Hijacking" it's a "Domain Helper", see: https://feross.org/shame-on-you-comcast/
The point of running a local unbound is indeed to turn on DNSsec validation. After that, it's the domains' fault for not using DNSsec. But at least your own stuff won't be tampered with, because you use DNSsec on your own domains, right? Right?
:-(
Coincidentally, a bunch of free wifi stuff blocks the root key queries unbound does at startup, resulting in nothing working until you get those records in there somehow (e.g. load_cache).
I was under the impression that dnssec's entire value was rather controversial? I'm not a crypographer so I had to skim https://sockpuppet.org/blog/2015/01/15/against-dnssec/ and I still don't understand it, but if
> In fact, it does nothing for any of the “last mile” of DNS lookups: the link between software and DNS servers. It’s a server-to-server protocol.
is true, then I'm ... pretty unsold on its usefulness. Anyone wanna ELI5 whether that's untrue or anything's changed in the last decade?
Rebuttal: <https://easydns.com/blog/2015/08/06/for-dnssec/>
General DNSSEC rationale: <https://www.icann.org/resources/pages/dnssec-what-is-it-why-...>
DNSSEC adoption graph: <https://www.verisign.com/resources/dnssec-tools/dnssec-score...>
Just yesterday, there was a talk by representatives from Ukraine and Germany about how and why they use DNSSEC:
<https://www.icann.org/en/engagement-calendar/details/icann-t...>
DNSSEC as designed is pretty useless. It was designed in an era where computers could not be relied upon to do cryptographic validation of something like DNS records, so that duty was delegated to DNS servers like the ones ISPs ran, where records could be cached.
There are also potential downsides (like someone being able to enumerate your subdomains, if you care about such a thing) and it's often switched off by default on operating systems consumers use.
However, it's the only thing that validates DNS records at this moment. That alone makes it worth enabling, in my opinion. For many domain owners, you only need to tick the checkbox in your registrar's domain control pane. If you use a modern DNS server, it's also one config flag and maybe copy/pasting a handful of fields away.
I'll gladly turn off DNSSEC the moment we come up with something better. DoT/DoH doesn't solve the same problem and no real solutions seem to have been proposed. That said, modernisation efforts are still being done. Smaller keys and modern cryptography rather than the massive RSA ones for instance.
I can't speak about the strength or weaknesses of the whole concept.
Out of curiosity I enabled DNSSEC checking beginning this year. I mostly noted nothing in my browsing. However, with 2 services I noted obscure problems that worked only partially. Some Jira plugins and Swedish railway SJ. I needed to excempt some of their domains from being checked to use their services. Of course you waste a couple of hours every time it happens because you don't think of DNSEC first when some web-based services behaves weirdly. There might be more requests not served to my browser, but I just never noticed or bothered to debug.
So not fit for general use. And even as an experienced developer it's not easy to understand why an exception would be needed. A bug in systemd? Incorrect data in DNS? A real attack?
Edit: I don't use standard DNS on port 53 at all. Blocked outgoing requests on that port. Everything DoH or DoT. Of course I need to trust those endpoints at least in repect to confidentiality. And most domains are unsigned. At least for those unsigned ones I need to just trust also in respect to integrity.
DNSSEC failures need to be louder. If websites that fucked up their DNSSEC would get the same error messages as websites that fucked up other parts of their security, people wouldn't be so eager to turn off security festures as workarounds.
Give SJ the option to turn off DNSSEC and their website will stay broken. Give people a "someone might be tampering with your connection" popup and they would start caring.
There are also other issues with DNSSEC that need working out, but I still believe it does much more good than it does bad. Leaving everything in the background with an easy-to-disable setting makes it next to useless.
> DNSSEC failures need to be louder.
Full ACK. As a part of this, DNSsec validation needs to be as close to the client app as possible and reasonable; definitely never off-host. (Caches can do additional validation, but that makes the error reporting complicated.)
corollary: the AD bit should never be relied on on anything actually on the network. Maybe on localhost, but even there it's questionable in UX results.
Note the linked post is from 2015. But also:
> domain-validated certificates remain insecure
...99% of the internet runs on DV certificates; that's what ACME & Let's Encrypt are at the end of the day.
(You might also note I didn't actually speak in favor of DNSsec, I'm just arguing that running unbound locally without having DNSsec validation enabled isn't particularly useful. I am, generally, in favor of rolling out DNSsec, but it certainly has its problems. It doesn't help against ISP tampering on unsecured domains, for example. DoT and DoH are better against that, if [and only if] you have a trustworthy cache to point them at.)
Adoption at the TLD is high as it is expected of the TLD operators but the second level is very low. So .org itself is signed but .org second level domains are just below 5% adoption overall. .com second level is around 4.2%. .net second level are a little higher at around 5.2%. So most people's answer would be a solid no. HN audience may be some of that 5%.
DNSSEC adoption rate means that this would be insufficient to detect (let alone prevent) that type of tampering for most domains. The correct solution is identified the the first paragraph of the post, which is to use a tamper resistant transport to the dns server (eg DoT, or DoH if you like everything being wrapped in http)
Which ISP was doing this? It's worth calling out the bad actors.
It's pretty much most Indian ISPs and in this case it was ACT Fibernet.
Wouldn't the list of ISPs not doing this be easier to make?
No.
There's a difference between what you're likely referring to, where the ISP's own DNS, which most people are using by default, returning particular results, and what the author is showing here where the ISP is actively tampering with outside DNS responses to insert incorrect responses.
Until now, I was not aware of any ISP doing the latter.
Only one ISP I've used ever did this (VodafoneZiggo) and that was over a decade ago, and only to redirect domains that couldn't be found to ad-filled "search" pages.
It's not universal, the occurance or this type of abuse seems to vary wildly from country to country
Want to ISP-less in India? :D At least ACT doesn't blanet throttle/fail uplaods of any sort like Jio does on its 5G – even LLM calls fail.
How is this not illegal... isn't what the ISP doing "hacking"?
Illegal, where exactly? In various western countries ISPs are by law required to block some websites. In other countries, ISPs ar eby law required to have escape hatches to be able to shut down the internet. In yet others, everything runs behind a firewall. Take your pick, I guess?
Very few, if any, countries makes this illegal.
Some countries have laws around Net Neutrality or Open Internet that can protect you from this, but even those countries have other legislations that can superseed that in specific cases. (e.g. hijacking DNS can be completely legal for some child protection laws, or to uphold laws that makes some web sites illegal)
The hijack is probably legally mandated. The site he was trying to reach is a torrent site.
Interfering in communication. If I understand correctly the ISP rewrote the author's communication with an external DNS/third party.
I’m using NextDNS over https and frequently experience connection freezes with both Spectrum and T-Mobile. I don’t know what they’re trying to do, but I’m not happy with the way they’re trying to block basic services.
Here is my self-documentation for home DNS setup: https://docs.zeropolis.net/doku.php/tech:homedns
Unlike OP, I have all my DNS route completely to my home DNS server, and only make changes to that container and my VM host during a "change window" meaning not during work hours. I block port 53 outbound entirely and have host entries for the two domains that my DNS server uses for DoH. If I had a better firewall on the edge, I would leave 53 open for my DNS container, but this workaround seems suitable enough for the home network.
It's best to treat your ISP's DNS as not being in your best interest, and actively against you getting your full internet speed and responsiveness.
Set an external DNS server on your router, or learn to run your own locally.
> Although your regular internet traffic is encrypted with TLS, your ISP can see the hostname during the TLS handshake (SNI).
https://support.mozilla.org/en-US/kb/understand-encrypted-cl... not anymore, with many (not all) sites.
Very few of the top 10k sites support ECH
https://divested.dev/misc/ech.txt
Deutsche Telekom did that too on my DSL line once, took some arguing to them suddenly 'do nothing' but it suddenly 'worked'
Sounds familiar to a Chinese
Reminds me of the Xfinity "Your personalized WiFi summary" emails. Despite returning their router, using a third party almost-layer-2 modem, my own openwrt router (with its own wifi, thank you very much), their emails seem spooky.
They have a counter of all of my devices, somehow (i mean... i guess their count is wrong, but it's infuriating that they try to keep track).
All of that could be done without "ISP was hijacking my DNS". DPI/SNI sniffing is more accurate and allows you to get more detailed statistics (eg. "20% of your bandwidth goes to youtube"
We need root DNS servers that support DoH/DoT.
No. That's not within the design function of the roots.
It's very much part of the design that roots and caches are separated. The security properties necessary between root and cache (authenticity) are fulfilled by DNSsec, not DoT/DoH (primarily privacy). And most TLDs are DNSsec signed, so this part is in fact OK. (Both give some level of tamper protection, in a different manner though.)
You're probably saying this because you run a local cache directly against the roots. If you need DoT/DoH, you shouldn't be doing that and rather forward your queries to a cache somewhere else.
(To make sure there's no misunderstanding here: DoT and DoH do absolutely nothing to guarantee the data you're getting is what the roots/TLD/owner published. It only ensures you're talking to the cache you think you're talking to. The cache can serve to you whatever it wants.)
my mobile network carrier blocks quad9
ISP can still see exit IP unless you VPN
Cloudflare Warp is handy, complete tunnel even if not as private as real VPN
since 80% of the web is on cloudflare it accidentally makes it faster anyway
and you don't have to use the cloudflare client, you can use Wireguard
Why not start by
1. contact the ISP and ask wtf they are doing
2. switch ISPs
How do you think that conversation will go? You phone them, get put through to the call center and surprise surprise its not covered by their script.
Switching too is funny. You are at the mercy of what isp ran a line to you. Starlink will fully cooperate with government too. They are a military contractor after all and also at the mercy of musk deciding dumb stuff.
well after going down a few levels of tech support you do get to someone that 'does nothing' while it the magically works
You make it sound like switching ISP is an option in every part of the world.
Yum, screenshots of terminal windows, which is just text...
Edit: oh wait, it finally loaded, 3x3MB images of text with scenery in the background!
Another reason to use a VPN you can trust, across your entire network and at all times.
A VPN just moves the attack vector from the ISP to the VPN provider, it doesn't protect you from this in any way at all. Better to use an ISP you trust, though that's difficult to find.
I don't trust any public VPN provider because they all start off by lying about what their product does.
But the isp sits between that no?
All traffic to the VPN is encrypted, including DNS queries.
All your ISP can tell is you are using a VPN.
What about TLS inspection?
Not unless they can break your VPN's encryption. Which, if your ISP can do that, you have bigger problems.
What about it? Unless they've hijacked your local cert store somehow (which corporate machines do), they can't see inside the TLS. They can detect that you're connecting to somewhere and probably infer that it's some VPN, but only the most repressive countries actually block VPNs altogether.
So a VPN you set up and run yourself.
Isn’t that the same as no vpn with extra steps? You are tunneling traffic from a computer you own to a computer you also still own. Third party vpn the whole point is I don’t run it.
If you set up a VPN endpoint on a Hetzner box, you're trusting Hetzner not to MITM your DNS queries. They're likely more trustworthy than your local ISP (although that leaves you more identifiable to folks running the servers or snooping on server-side traffic, and Cloudflare will hate you). That threat model tradeoff is good for many people.
Given the OP use case, you are going to get a letter from Hetzner at some point.
Not if they are only using it to route the DNS traffic (though in that case a cheaper VPS might be better), which will work if the ISP is only poisoning DNS replies and not (which is often the case) blocking traffic to the target servers entirely.
The problem with this is you are adding a moving part to your DNS lookups. It might be worth adding two on separate VPS hosts and have your resolver fall back to the second if the first goes down.
Another, usually more minor, problem is that some services (like CDNs) use the source of your DNS query to modify the response so you get sent to a topologically near-by end-point. This means you might notice such services being slower as you are sent to a PoP near your VPN host not one closer to your actual location.
Oh well OP was doing shady stuff ... then yeah they should continue with using VPN provider. VPN providers are shady AF, I don't think there is VPN provider I would trust more than my local internet provider.
Well that wouldn’t be running your own vpn. Hetzner is. They’d let you do this? Seedbox is against their tos hard to imagine they’d be a good choice to run a vpn for your linux isos.
> They’d let you do this?
Can't speak for Hetzner, but considering VPN use in Russia and China - I'd imagine most EU\US vps providers have someone's 3x-ui, remnawave or naked v2ray installed.
Plus - how are they going to detect your remnawave node sitting on 443 port? For them it's just some traffict.
And if you are worried about the amount (torrents) - you don't really need to torrent though them. You just need an access to a torrect tracker to get your magnet link and meta info.
Most default configs for 'ray have torrents locks for this very reason - you communicate with other people (seeds\peers) directly.
>Plus - how are they going to detect your remnawave node sitting on 443 port? For them it's just some traffict.
What about all the outgoing traffic? That's going to be hard to mask unless you have another VPN for outbound
But what's wrong with my VPN talking to, say, youtube (blocked in Russia) or some torrent site (for example to fetch RSS feed of new releases).
Fedora, for example, provides torrent access (including RSS feed) for its releases: https://fedoraproject.org/torrents/45/
The story ended before it even started... What are the machine with the wrong IP doing? Is it just forwarding the request, or what is happening?