I seem to remember a paper a few years ago that (IIRC) tested this by setting a very low TTL (like 60), changing the value, and seeing how long they continued to receive requests at the old value... and most updated within the TTL, but there were some that took up to (I want to say) an hour. I'm probably getting bits of this wrong though..
The violations in that paper that are important are those that have increased the TTL. Reducing the TTL increases costs for the DNS provider, but isn't important here. The slowest update was about 2 hours (with the TTL set to 333).
Of those that violated the TTL, we don't know what portion of those would function correctly with a different TTL (increasing the TTL indicates they're already not following spec). So I wouldn't assume that increasing the TTL would get them to abide by your requested TTL. They're following their own rules, and those could by anything.
Considering how common low TTLs are... you're worrying about a DNS server that's already potentially causing errors for major well known websites.
It is important to note that this study used active probes asking selected recursive resolvers around the world.
From my own experience when changing records and seeing when the long tail of clients stops calling the old addresses (with the name), it is a really long tail. An extreme example that lasted almost six months was a web spider that just refused to update their DNS records and continued to request websites using the old addresses.
Is there a lot of custom written code that does their own DNS caching? Yes. One other example is internal DNS servers that shadow external DNS. There is a lot of very old DNS software running year after year. Occasionally at work we stumble onto servers which are very clearly handwritten by someone a few decades ago by people with only a vague idea of what the RFCs actually say. Those are not public resolvers of major ISPs, so the above study would not catch them.
Naturally if you have a public resolver where people are constantly accessing common sites with low TTLs then issues would crop up quickly and the support cost would get them to fix the resolver. If it's an internal resolver inside a company where non-work sites are blocked then you might not notice until the company moves to a new web hosting solution and suddenly all employees can't access the new site, an hour later they call the public DNS hosting provider, the provider diagnoses the issue to be internal of the customer's network, and then finally several hours later the faulty resolver gets fixed.
> An extreme example that lasted almost six months was a web spider that just refused to update their DNS records and continued to request websites using the old addresses.
It may have been Java client that was not restarted. At least for older versions of Java the default was to cache result forever.
Yep, older java versions had some ridiculous caching of both positive and negative DNS responses. That was some weird problem to troubleshoot. We ended up writing our own caching then, back in Java7ish. And the first version of our DNS caching was broken and promptly triggered load alerts on 2 DNS servers of our operations team by issuing ... a lot of DNS queries very very quickly :)
> An extreme example that lasted almost six months was a web spider that just refused to update their DNS records and continued to request websites using the old addresses.
Could it be that it wasn't using DNS at all? Just hardcoded the ip address?
I did find this paper: https://labs.ripe.net/Members/giovane_moura/dns-ttl-violatio...
The violations in that paper that are important are those that have increased the TTL. Reducing the TTL increases costs for the DNS provider, but isn't important here. The slowest update was about 2 hours (with the TTL set to 333).
Of those that violated the TTL, we don't know what portion of those would function correctly with a different TTL (increasing the TTL indicates they're already not following spec). So I wouldn't assume that increasing the TTL would get them to abide by your requested TTL. They're following their own rules, and those could by anything.
Considering how common low TTLs are... you're worrying about a DNS server that's already potentially causing errors for major well known websites.