Introducing the New Google Index
You may have heard talk about a new and improved Google index called Google Caffeine – but we had no idea when this new Google index would be released. Turns out, it has been released now and is slowly making its way to a Google data center near you. What IS Google Caffeine? And more importantly, what does Google Caffeine mean to YOU? Keep reading and let’s talk about this…
Old Google vs New Google
Before we get into talking about the “new” Google – Google Caffeine – we need to back up and talk a bit about old Google and how their search index worked.
Early in 2009, I did a post about how Google indexed web pages. In short, indexing was done in “layers” or stages. First, they would find your url, then over time, it would be released into the main Google index.
Ever done a site:YourWebsiteNameHere.com search on Google only to find that Google DID find your new site or new web page, but when you search for your keywords, your site or page is nowhere to be found in the main Google index?
This happened because your site (url) was stuck in transition – meaning Google found you, but needed to wait until their main index updated to ADD you to the main index.
Indexing happened in steps or phases. When your site or url was added into the main Google index all depended on how often Google updated their main index.
Updating The Google Index – Some History
Back in 2000 and earlier, Google updated their main index about 1 time every 4 months. So yes, that means that webmasters could very possibly have to wait FOUR MONTHS to see their site/url start ranking within the main Google index.
FOUR MONTHS! (feel a little spoiled now that we can complain if we aren’t indexed within a WEEK?)
Then, in 2001, this “little” event happened called 9/11. Google struggled to keep up with requests for fresh and new content relating to this horrific event – in fact, they COULDN’T keep up. Google realized that something had to change.
Google then went to updating their main index once a month. And this is a BIG process. Google is not one single main index – they are actually comprised of numerous data centers all over the world. This means that you and I could search for the same phrase…and get different results. It is a major process attempting to get all data centers to show the same results.
So, to summarize, getting Google to find and release your site/url into their main index was kinda like waiting on a bus.
You wait for the bus (build your site/url and leave something to help Google find you). Finally, the bus shows (Google crawls/finds your content) and you get on the bus. Now, you wait until the bus drops you off at your destination (your online content gets released into the main index).
Now, that is a very simplistic way to look at how Google finds and indexes new web pages, but that is the basic concept. Your “bus ride” (ie, how long it took your new web content to SHOW in the main Google index) all depended on how soon until the next monthly index update.
Thing is, this is now how “Old Google” did things…. but that has all changed now that Google Caffeine is here.
Google Caffeine – What Is It?
First, let’s explain some meaning behind the name “Caffeine”.
Old Google was decaf…. New Google is caffeinated. Old Google = slow with no kick… New Google=fast and kickin’.
Google Caffeine is not an algorithm change… meaning it is not something designed to change HOW Google ranks web pages (ie, ranking factors…ranking signals…). It’s not an optimization thing, for the most part.
Google Caffeine IS a change in the way Google finds and releases web content into the main Google index.
It’s faster… claimed to contain 50% fresher results.
Instead of being a “phased” indexing process… it is now more “real time”.
It’s the difference between waiting on a bus… and saying “Beam me up, Scotty”.
Old Google vs New Google
This is a BIG difference in the time it takes to get new content into the main Google index!
Let’s work with an example based off my own understanding of what is going on.
Google Caffeine Indexing Example
From what I understand, new content – fresh content – on a topic will be found much more quickly AND will be able to be FOUND in the Google index very quickly (like within seconds).
Now, in essence, this does not affect your rankings in Google – just how fast your content is found AND released into the main Google index.
But let’s take this example of how Google Caffeine could affect your Google rankings.
Let’s say you have a Squidoo lens about growing apples.
It sits pretty day after day in the Google index, ranking well, bringing you some traffic. The competition is not tough so you don’t have to do much to maintain your top ranking.
All is good in apple land.
Then, one day, this major news story breaks about a serious health threat due to apples – specifically the way apples are grown.
BAM – the Google results for your keywords will now be saturated with “real time” news and updates about this situation. They will rank above you (from my understanding).
How to counteract it?
If this happened to me, based on the way I understand the new Google Caffeine to work, I would totally update my lens with new content and info relating to this apple situation. I’d then make sure it gets “talked about” on Twitter and/or other social networks so my apple lens might be part of the new “fresh” content that Google wants to offer its users.
Ya see, that’s what it is all about… Google wants to be able to offer it’s users the newest, freshest, and most up-to-date info on the topic they are searching for.
I think that is pretty cool.
Consumer Expectations From Google
User/consumer expectations of Google have changed over the years.
Personally, I can say that even though I used the internet back in 2001 when 9/11 happened, it wasn’t my first thought as to where to get information. I still relied on primitive things like newspapers, TV and news…. radio. It never crossed my mind back them to hop on the internet and get current information.
Today, totally different story. The FIRST place I go for information is the internet…. specifically Google. I have a sneaky feeling that I am not the only one who has evolved this way.
Google is evolving to meet our needs and our expectations as consumers.
Is Google Caffeine Showing For YOU?
While Google has said that Google Caffeine is now live, I don’t believe that it has made its way to my data centers as of yet. There is this site where you can compare “Old Google” search results right next to “New Google Caffeine” results (with no personalization) and when I do that and compare to my own Google results – I believe I am still seeing “Old Google” results.
Note: The site to compare old vs new is here.
I imagine this will change over the coming days as Caffeine makes it way around the globe.
How Will Google Caffeine Affect YOU?
Please keep in mind that this is all my opinion and feeling after reading a LOT about all this “New Google” stuff. NO ONE really KNOWS what Google does, ok?
I am thinking that the big sites might have some issues. BIG sites – think Amazon.com, etc… BIG sites. There sites are full of catalog type pages that I don’t imagine they do much link building too nor do they do a whole lot of social marketing for.
But, as I poke around, it appears that when Google claims that Caffeine will be a BIGGER index, they just might mean it. Comparing site: type searches for a lot of sites, I see MORE pages indexed in the “New Google” index than in the old.
In fact, the official Caffeine announcement on the Google blog puts Caffeine into a crazy perspective:
Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second.
Three miles taller EVERY SECOND! Holy cow! That statement alone was a WOW moment for me!
I am in agreement with this post that says they believe news sites and blogs will come out on top with Google Caffeine – and Matt Cutts from Google has apparently confirmed this (and needless to say, this pleases me )
Fresh relevant content released into the Google index faster… that’s what it’s all about.
Will Your Ranking Change In Google Caffeine?
Maybe, maybe not. As I’ve felt about the “Old Google” index, I feel the same about the “New Google” index – the more competitive the query space, the harder you need to work to get rankings and KEEP rankings.
If Google is “hungry” for content and they don’t have much to offer, your rankings should be more stable. If your keyword space is really dynamic (ie, competitive), you’ll need to work. I don’t think that changes with Caffeine – unless some timely news event happens or the trend suddenly changes.
If you’re looking for a keyword to work with that you don’t want to exert that much effort into, check to see what Google has found in the last 24 hours. Are they hungry? Or are they finding a buffet of content to return for a user search?
Will You Need To Constantly Update and “Freshen” Your Content?
Again, this is a maybe-maybe not type answer. If your rankings and traffic are fine, then you should be able to continue with business as usual. However, if you see major changes in your rankings, you’re probably gonna want to get to work.
As I talked about in my Google Tiger-Tail Effect post back in December, Google is trying to offer their users the most recent content and comments on a hot topic as fast as possible. Those scrolling Twitter links on a search were a BIG hint as to what direction Google was trying to go in order to please their search engine users.
We, as searchers (not as marketers…lol) want the most timely and newest information on a topic we are searching for. If we do NOT want “new” results, HOW we search will reflect that.
For example, today a BIG thing happen with USC (University of South California) – it is MAJOR. All I have to do is type “USC” into Google and I’d hopefully get the “real time” information I am looking for.
As you can see, all I have to do is type – usc – into Google and the info I am looking for shows right at the top – pushing the usc.edu site down a notch.
But if I’m looking for USC football stats from the 1995 season, I, as a searcher, would not type only usc into Google – I’d type want I want…. usc football stats 1995 (or something like that).
And when I search like that, the Google results change to reflect what I am looking for.
Summing It All Up
What Google Caffeine will mean to YOUR online activities will vary from person to person and from site to site. I think trusted news sites, blogs, social sites, and user-generated content type sites might come out well with Caffeine. Larger “static” older sites might struggle, but that will remain to be seen.
As always, getting relevant and fresh content out on the web AND getting people “talking” about it (ie, linking within conversation) will be very important – now more than ever. The degree this will be necessary for YOU will depend on the dynamics of your query space.
In short, Google may have gotten faster and bigger – but did they really change THAT much?
Time will tell.
Matt Cutts Video About Google Caffeine and More Resources
I’m going to leave you with a list of content that I’ve read about Google Caffeine and a video of Matt Cutts talking today at SMX Advanced (note: the video was hard for me to hear – had to turn my volume way up – but it’s really worth watching and listening to)
I’d love to hear your comment and feedback in the comments area. If you need more information about Google Caffeine, well, just search Google