The pingdom alerts started popping into our Slack channels just after 9 this morning. Sites that are reliable started going down. We had 3 within the space of a minute go down. All up we had about 5 client sites affected. As at time of writing one site is still down.
The DNS Problem
We quickly established that these sites had DNS issues. I checked their nameservers and two were using partnerconsole.net and one was using netregistry.net . When I checked the partnerconsole.net nameservers (TPP Wholesale) I nearly fell off my chair.
Not a good feature in a name server. Then I did a whois on partnerconsole.net to find out who was doing its name serving.
So then I pinged the netregistry’s name servers.
Oh dear. There was nothing on Twitter or any announcement from anyone we could find so we put it out on Twitter. Then a short time later Netregistry called it.
Partnerconsole.net has since changed it’s nameservers away from Netregistry.
You can get updates on the outage here: status.netregistry.com.au ironically or perhap conveniently their .com.au does not use netregistry.net name servers.