A Day Without .IO TLD DNS Issues and Cloudflare

Yesterday morning, I started getting down alerts for this blog. Pingdom alerts include the message: “Cause: Non-recoverable failure in name resolution”,

From the start, to make troubleshooting easier, I misplaced Cloudflare as I had a similar issue with them a year ago and was able to solve it by temporarily deleting it. However, upon checking, the issue was not related to Cloudflare and soon resolved on its own. Although Cloudflare’s support pointed me to hn discussion thread,

now, this morning GTMetrix I noticed an increase in load times. I thought, “Oh yes! I should revert the nameservers to Cloudflare.Before I did this I captured the graph on this page that shows how the response times doubled cloudflare Removed. As Joni Mitchell put it in her song, Big Yellow Taxi: “…you don’t know what you got ’til it’s gone” and that goes for Cloudflare, but also for DNS resolution of my domain . laugh out loud

uptimerobotto like psdiAlso shows a jump in response times without Cloudflare….

Dashboard Uptime Robot

Released on 20 September 2017 due to .IO TLD

There are issues with resolving intermittent global DNS records on .IO domains, affected technical domains, and .io startups. One such affected startup was dns spy who is the founder matias jenniar Provided me with some details about the cause, he noted:

,IThe .IO TLD uses 7 different nameservers for its top-level domain; a0.nic.io, ns-a3.io, c0.nic.io, ns-a2.io, b0.nic.io, ns-a1.io and ns-a4.io. ​—2 of those nameservers, ns-a2.io and ns-a4.io, started misbehaving and replied with an NXDOMAIN result instead of returning with the correct set of nameservers for the domain you were requesting started giving Essentially declaring that the domain you were requesting did not exist.

The problem is that NXDOMAIN is a valid DNS response, which can be cached. So a DNS client doesn’t retry its query on a different nameserver, it got an answer and will honor it: The domain you’re trying to access doesn’t exist. As far as I know, there has been no official communication from the .IO Registry, so we are left only to speculate. Was this related to the recent DNSSEC key increase? Was it a targeted attack? Was this a human error? software failure? We don’t know.,

With yesterday’s (and past) .IO events, we can only hope that raising awareness will help ensure that future issues, made more transparent or better, can be avoided.

Music Reference: Big Yellow Taxi by Joni Mitchell

Leave a Comment