Temporarily Unreachable
UPDATE: We have fixed the issues with our clients. It turns out there were some configuration issues with the DNS but they had been like that for a very long time. It seems that Google on October 8 started to pick up on these sorts of DNS issues. If you have one that started on October 8 please let us know.
Transcript
Welcome back Rankers! So, what’s happening with Google’s new Penguin 4.0 rollout? Personally, I think it’s still happening, as there were some reports over the weekend that claimed the Penguin rollout had finished. Google didn’t actually say that on Twitter, rather they said that anybody that had penalties from previous Penguins should now be okay. We’ve still seen over the past few days things still bouncing around a lot. Especially in competitive areas. Now maybe that’s the new norm, things bouncing around in competitive spaces. Certainly we’ve observed sites being on page one, one day, then disappearing, and then returning the next. In my experience over the years, that’s usually a sure sign of an unfinished Google update, or the data centres having incomplete information.
However, we have found another oddity.
DNS Issues
We have found DNS issues in Google Search Console on a couple of clients. If you don’t know what a DNS issue is, simply go to your ‘Crawl Errors’ section in Search Console and you will see Site Errors at the top.
Underneath that, you will see a tab that says DNS. If you have DNS issues, they will be listed front and centre right here. Now, we have two clients that are experiencing DNS issues, which in itself is a little strange. I’ve only experienced one past client with this problem, so to have two sites, on the same day, totally unrelated as one is a .com and the other a .com.au, with one being hosted in the States and the other down the road here in Noble Park, is very unusual. We’ve done all the DNS checks but haven’t found problems.
The problem with this particular error, though, in Google Search Console, is that Google is saying that it can’t actually see the site. Google states that the sites are unreachable. On the eighth of this month, both site’s crawl stats dropped to zero, as Google said they were unreachable.
Now obviously they aren’t unreachable, as we can go to them, Screaming Frog can go to them, and we’ve mimicked the Google User Agent and gone to them. We have used the Google Page Speed tool and fetched the pages. I’ve used a structured data testing tool and fetched the page. So we’ve used all these Google tools that can actually fetch the pages, but Google is still reporting as recently as five minutes ago, that they are ‘Temporarily Unreachable.’
Not Just Us
So you might say, “Jim, why don’t you speak to Google?” Well I did, and apparently Gary Illyes and John Mueller have not seen this particular error. John said, “I’m sure lots of people have crawl/DNS errors. Errors aren’t bad, sometimes one-off, sometimes avoidable.” These are bad because we can’t fetch any pages for these clients, and their rankings have started to drop over the past few days.
I did Google the problem and found another lady called Cathie who was experiencing similar problems on the same day. I’ve never met Cathie before, so I picked up the phone and called her. I asked her if she had found a solution. She said they moved the whole site to the root domain and removed the www’s. That’s a hell of a fix.
In DNS terms that would suggest your A-record or your C-name is incorrect, because the root domain works but the sub-domain doesn’t. Problem is that we’ve checked all those things and they aren’t the culprits. But Google Search Console is having an issue with it.
The other thing Cathie tried, and a big thank you Cathie for taking the time to talk to me, was to manually enter the Googlebot name into the robots.txt file. So rather than just say user-agent-wildcard, she actually put in user-agent:googlebot, or user-agent:smartphone, whatever the bot’s name was. She found that started to work.
Cathie runs a site called babygiftbox.com.au and does Google Shopping. She was having trouble getting her merchant feed to synchronise with Google Adwords because Google was seeing her as temporarily unreachable.
The Solution
One fix if you are having this problem is to move domains, which in reality is a ridiculous fix as it appears as though the Google Search Console is having an issue, or something with Google is having an issue. I can’t confirm it, and Google is denying it. I went out on a limb with this, but when you have three sites and the data tells you it happened on the same day, and all experienced the same issues, it’s a little fishy. None of them actually has those issues in the real world.
Cathie said she spoke to her hosting company, she spoke to the people who handle her DNS, she spoke to her developer, and nobody had changed anything. All these sites are on different platforms too.
If you are experiencing this problem, please let us know, as we’d love to find out more about it. If I find a solution in the meantime I will let you all know, so watch this space. Incidentally, for bloggers following this, we now have the bloggersSEO support group now open on Facebook. If you are a blogger on WordPress, then head across to bloggersSEO.com and you’ll find the link to the support group.
I’ll see you all next week. Bye for now.
Jim’s been here for a while, you know who he is.
