Google Crawl Budget. Popularity Not Backlinks
Crawl Budget Explained
Welcome back Rankers! There was a story a couple of weeks ago about ‘Crawl Budget’. There has been a lot of talk around Crawl Budget, hence Google doing a story on it on webmasters.googleblog.com. I’ve done shows about this before and have spoken about it, but this is the first time I’ve seen something in print from Google about Crawl Budget.
Google Admissions
Some of the things we’ve observed over the years they are now admitting to. A couple of things we know such as if your site moves to HTTPS you will see a spike in the crawl rates. That’s what we can see here. We are in Search Console-Crawl Stats and we can see this example site went to HTTPS and Google has subsequently been crawling the site a lot more as it gets used to that ecommerce site’s HTTPS pages. So there’s that situation where Google will come in and crawl your site a lot more and then there’s the situation of you getting a speed increase on your site.
This is one of the reasons why it’s so important to have a fast site because as you can see this site here, pages were loading on an average for Google, once again this is for Google and not for a human, were loading about 840 milliseconds each and now they’re down to just over 200 milliseconds. You can see there’s a distinct line between Google’s crawl rate and the speed of the site.
Popularity, Not Backlinks
The third one that they mention and lots of people gloss over this, as I know a lot of people think this is referring to backlinks, but I don’t think so and I’ll show you why. So Google actually says that popularity is something that will make Google crawl your page more. So here it says, “URLs that are more popular on the Internet tend to be crawled more often to keep them fresher in our index.” So what they are saying is that popularity is a measure of the bot coming out and I would further go to say it’s a measure of a new form of backlink. Popularity used to be backlinks, but Google has been very careful there to not say backlinks but popularity.
Case in point. This is a Melbourne client and this represents a national news story for this particular client back on the 14th November last year. You can see on that same day Google crawled a bucket load of pages. The reason for that is that Google saw it was more popular as more people went to the site. Incidentally, that spike I showed in analytics was organic traffic. We had a brand spike in organic, so more people were searching for the brand. All of a sudden, the brand becomes more popular, more people are getting to the site from Google on that particular 48-hour period, and we have a subsequent spike in the Googlebot accessing pages and crawling pages.
Build It and They Will Come
So when we talk about popularity that’s why we are so big on not talking about backlinks or building backlinks and those sorts of things. We like to focus on building audience, because if you’re building audience, you’re increasing your popularity, if you’re building your brand you’re increasing your popularity. Increasing your brand popularity is the most important thing. It’s not backlinks because you’ll get those through increasing your brand popularity. Yeah, backlinks are important, but the way that you get them is far more important. Focussing on that brand, you’ll start to see spikes and traffic increases. Firstly from the Googlebot and then increased rankings. I’ve seen it happen time and time again, and every time we do this we see massive spikes.
I’ll give you another example. We just moved to new hosting last week and you can see here we were quite slow, loading at 1.5 seconds and now we’re down at 86 milliseconds. That’s Googlebot page load time. We are starting to see an increase in crawl. So that happens every time and that can only be a good thing.
It’s a good article to go read that one from Google as it also talks about eCommerce sites and things like facetted navigation. The reason we are always trying to use canonical tags, noindex tags, use nofollow tags, all those sorts of things, that article explains it really well.
Hopefully that’s helpful and we’ll see you all next week. Thanks very much. Bye.
Jim’s been here for a while, you know who he is.
