Thinking About Starting Your OWN Blog? If YOU'D like to learn how to make money blogging with affiliate marketing like I do, feel free to read my Free Blogging 101 E-Course here.Help For Bloggers | Get Daily Pinterest Tips
Get Free Training, TWO Free Blogs, and Free Tools
So tonight I am minding my own business and sitting on Twitter trying to catch up on some cool Tweets and industry info. Suddenly a tweet from Matt Cutts to ME pops up on my stream. (In case you don’t know, Matt Cutts is the head of web spam at Google). He let me know that he left a comment on my Google Quality Raters post to clear up a misconception. It was something *I* said in my post – and yep, he totally debunked me…lol! Hey, when you’re wrong, you’re wrong – so I want to clarify and stand corrected. Here’s what Matt Cutts clarified for me.
In that post about Google Quality Raters I was quoting pieces of an interview with Scott Huffman (Google’s Engineering Director). Mr Huffman was asked if ONE Quality Rater can affect the rankings of a url. Mr Huffman said No (in more words than that, but “No” is the gist of it).
I then went on with my post and *I* said:
“Now this makes sense to me – ONE rater can not cause a rankings change. However, I do believe that if a certain percentage of raters mark one url as spam or non-relevant, that it does throw up some type of flag in the system that can cause something to happen to that url. Now I naturally do not KNOW this, but I get that sneaky feeling.”
According to Matt Cutts, my “sneaky feeling” is wrong.
This is the comment Matt left in the comments:
So- there it is – according to Matt, I am wrong (well, my “sneaky feeling” is wrong).
Quality Raters can not directly change the ranking of a url.
All they do is is review potential rankings caused by a test algorithm change. Based on their feedback (ie, their ratings), Google can tell if the test algo makes things “better” or “worse” in the SERPs. If it is deemed “better”, that test algo is released out into the Google Sandbox where a small percentage of live Google traffic gets the test algo results.
If that sample of live traffic shows that the search results are “better”, then the algo is released out to all of us.
And THEN your rankings could very well change.
That’s the short version from how I understand it. If you want to understand the complete process of this, read this post: How Google Makes Algorithm Changes
So, let’s continue the story…
So Matt Cutts (@mattcutts) tweets to me (@potpiegirl) that he left a comment on my blog.
I thank him and then ask him a question:
“Would multiple spam ratings cause an internal Google employee to manually review that url?”
To me, that makes sense. If many HUMAN reviewers all flag a site as spam, wouldn’t it make sense that someone internal with Google would check that url out?
His answer to me was short and sweet.
Here’s the entire Twitter conversation:
Naturally, his comment on my blog here at PotPieGirl.com sparked more questions from others on Twitter.
Here’s some more Q&A regarding his comment about Google’s Quality Raters and Google rankings:
Which amusingly led to Matt Cutts making a “note to self” (uhhhh… a “tweet” to self? lol!)
Nice to have a video about all this – or at least something about all this – thanks, Matt! And thanks for taking the time to stop by and “debunk” me 🙂
I will say this – it does make sense to me that the data gathered from the human Quality Raters should not be “mucked” with. If you go in and start manually changing things, you don’t really know if the algo ever gets better or worse.
Edited to add: Matt came back to answer some questions others were asking him the comments area of my post.
Here is more from Matt:
So, there you have it…
What do YOU think?
I Need To...
- Learn How To SEO My Blog Posts
- Learn How To FIX My Pinterest Results
- Learn How To Track My Blog Posts for Better Results
- Learn How To Earn MORE From My Blog Posts
- Learn What Works NOW for Bloggers
- Learn How To START Blogging for Money
- Join PotPieGirl's Affiliate Program
- See What Tools PotPieGirl Uses
&
&
“according to Matt” is the operative word here. I am skeptical, in fact I do not believe it.
Agreed. Google Human Raters have nothing to do with the webspam team… which is why the leader of the webspam team is releasing statements about them…
Now that’s an interesting story 😀 Although what Matt said was really logical but I also think that he will never tell the whole story as it might backfire on him. But thanks for sharing this information Jennifer, you’re always on the news!
Thanks, Duy!
Above all and regardless of our opinions, Matt didn’t have to say anything at all, ya know?
Jennifer
lol which begs the question… why did he?
Kinda wondered that myself… thought this was all “old news”….lol!
Matt has no reason to outright lie here, or go out of his way to do so. Obfuscating the truth is one thing, you can see that when he dodges a question – but to call the guy a liar is a bit of a stretch.
At this junction, I’d say “debunked.”
Quality raters are one department – and their work DOES inform the webspam team (or vice versa) so that the algo gets tweaked to weed out more spam…the Q/A team is simply there to make sure the changes are good and the algo is fit for production.
But of course this also means the *indirectly* the Quality team *does* affect the algo, in the sense they give a “yes or no” to whether the change is better or worse.
That’s how I understand what Matt’s saying. No controversy there…a better question to ask is why brands are so favored vs. small business like affiliates (the quality affiliates I mean)…?
I agree James…and not gonna appreciate any name calling.
The Quality team gives the algo team feedback to let them know if the algo change has potential and is worth further testing in the sandbox. I guess it comes down to “yep, this tweak has potential, let’s keep testing” or “we totally missed with this algo tweak, let’s can it”
Thanks James!
Jennifer
Google never lets information go to waste. I do not believe Matt Cutts or anything Google says because they are self serving and very cagey about info they release for fear someone might game their algo.
It’s a safe bet that if raters flag a site for spam they turn it over to the spam team. So what Matt says is true in a sense, the raters or should I say ratters don’t devalue a site, they hand over the data and let someone else do the dirty work.
“It’s a safe bet that if raters flag a site for spam they turn it over to the spam team.”
No, that’s a very bad bet, because it’s not true. If search quality raters rate a site as spam, it’s not sent over to the webspam team. Please see what I replied to Jon Cooper: you don’t want to spamfight on the data you use for metrics, or else you’ll get skewed metrics.
I really appreciate you taking the time to clear these things up, Matt. Thank you.
Jennifer
Thanks for the clarification, Matt – the fact is that people will believe what they want, but taking the time to talk about it is much appreciated.
And for Jennifer for posting this. Fist bump. 🙂
I also appreciate your taking the time to weigh in on this, Matt.
What I find confusing are the semantics in play, here. If multiple spam flags are levied against a URL in an algo test, it could still affect the site’s rankings, depending on whether that algo change is implemented. So… can you really say unequivocally that it CAN’T affect a site’s rankings? Even if the URL isn’t handed over to the webspam team , it COULD potentially affect the site’s rankings, no?
Plays on words, but considering the confusion around the role of human raters, I think it’s an important distinction.
This is an absorbing thread and my thanks to potpiegirl.com for initiating and Matt for engaging. Just so I can feel good about making those student loan payments on my Jesuit education…Google quality raters do not impact search results, they can influence algorithm changes at Google, algorithm changes at Google influence search results. Wouldn’t Aristotelian syllogism conclude that the raters do have a distant influence on organic results?
MattCutts said: “you don’t want to spamfight on the data you use for metrics, or else you’ll get skewed metrics.”
Fine, but what’s stopping Google from using this the after the algo goes live?
NOTHING. See my nickname
Matt,
I appreciate your responses. Its great to hear your side of the story on this post and others. I tend to keep things simple and after doing seo for years it seems Google tends to do the same; provide a good search experience (relevance from credible source), provide value on your site/blog, ie unique content like this post, and everything else is fluff & filler. But hey, I understand all the speculation, cloak and dagger, yada, yada, its always been around, and will always be around.
Jennifer,
thanks again for this post. I almost laugh at all the speculation and interest this post has generated. “Note to self”, write about Google Raters so my blog can get the same attention as potpiegirl. I think your post has generated more back and forth discussion than everything this month other than Tim Tibow. ; ) Well done
Thanks a lot for very interesting info about quality raters and comments from Matt. A lot of things cleared up.
So, to clarify, is Google saying they only use raters to evaluate search results from a potential algo change and never from the present/currently used algo? Even if the answer is ‘yes’, I think it still safe to speculate that the parameters raters use are very similar to those used by the “spam team”!
Or maybe Matt would care to enlighten us further?
There is an interesting insight here.. obviously from this discussion the feedback from “Search Quality Rater” team can not cause a site to be manually flagged in any way.
However, is there a “Web Spam Rater” team in existence? Is there another group of manual reviewers that analyse the results in a similar way, but can manually flag sites?
I would love to know how a “Thin Affiliate Penalty” gets applied to a site. This is where all the SERPs are dropped -50 places. I know this exists – I have seen it happen to my sites and others. Once it happens no reinclusion request or changes to the site have lifted this penatly
Does a manual “Web Spam Rater” team apply this kind of penalty?
More info here http://www.pickledshark.com/thin-affiliate-site/
Thank you Jennifer for getting this discussion going with Matt Cutts. You are encouraging and your tips on surviving Pandas is great intuitive down to earth practical psychology.
There is Matt Cutts and his web spam team and then there is Google: the Google that own Google.
Matt Cutts is in a difficult position. He works for Google with web spam. The people he is fighting are not decent people. They are a nasty crowd that most of us decent folk want banned and buried, not just sandboxed. When Matt is trying to help us and be informative he HAS TO be cagy and “lie” by omission because those evil nasty web spammers are hanging in every word he utters to get one up on Google.
For us decent people doing a decent days work to earn a decent income, Matt Cutts is not open enough. That is different from lying.
My gripes with Google is when I becoming collateral damage in Google’s business strategy, which I believe is messing up their search algorithm.
Very true Charles!
No matter WHAT Matt Cutts says, there is the group of us trying to learn from it to be better…and there’s the group that wants to find how to game it.
Saying too much can be very dangerous, I suppose.
Jennifer
“My gripes with Google is when I becoming collateral damage in Google’s business strategy, which I believe is messing up their search algorithm.”
Totally agree … they are forced to create a steady increase in income, but as google’s market share is already as huge as it can be, they are forced to compromise their intial idea of the “perfect search engine” in order to produce new income streams.
Misdirection? Where’s my tinfoil hat?
awesome conversation Jennifer.
my opinion? CYA and Matt will never tell all that he knows 🙂
Sometimes it’s so hard to sift through and sort out all this information……it all becomes very confusing.
I simply try to make the best site possible using all “white hat” strategies, send it out there, and let Google do what they will with it. I know this sounds like a very “hands off” approach, but it works for me. I do not like to stress too much about this or that…….I just follow my gut instinct as I go through the process of building sites and performing SEO and ask myself – “Is this the right thing to do?” If the answer is yes, then I’m probably on the right track and I don’t worry about anything else.
I try to stay on the happy side of life at all times!
Thank you Jennifer and Matt for this information…….
Amen John! Preach!
Wow – Reading this series of posts I was first inclined to disbelieve Matt – I mean, multiple spam ratings do NOT send up a flag? How could I believe that for a minute? But after digging in and reading more closely, the “mucking” argument makes sense to me too, and i’m satisfied with the official answers.
However, if several Quality Raters mark a site as spam, even though Matt says it’s not “sent over to the webspam team” he doesn’t say that something else might not happen to it – another team perhaps – someone MUST be doing something with them, right?
Matt is in a tough position, and while I believe he’s never outright lied to folks, it’s pretty obvious when he’s being cagey and obfuscating facts – He’d make a great politician.
This stuff is fascinating… Thanks for these posts, PPG 😉
Agreed, here… at the beginning of the leaked Human Raters guide, it does say that these URL raters will be ready for other rating jobs once they can do this successfully. It seemed like Level 1. I wonder what those other rating jobs are.
Hiya Scott…
True, there is most likely more to the story and things NOT being said.
BUT, based on what I had posted, Matt’s clarification was that Quality Raters cannot directly change the rank of a web page.
I personally find it hard to believe that NOTHING is done with that data.
For example, what if a rater comes across a webpage that is spreading a virus? Nothing is done about that (ie, it’s just left out there in the wild)?
Sadly, this happened to me last night… I hit a web page and my computer went INSANE. Spent most of the day trying to get back to right. So annoying.
My point is, that is so much data accumulated from HUMAN eyes… I can’t believe nothing is done with it. Well, it goes back to the algo team – perhaps the human feedback leads to footprints/things in common amongst the spam ratings – and then churned out in the NEXT algo upate…and then naturally, there are casualties…and on and on.
I’m still not convinced, but that doesn’t matter. The man has spoken so I’ll accept it and move on (ok, I’ll still ponder it…lol).
Matt didn’t have to stop by and say ANYTHING. Heck, there’s lots of bad info all over the web, why bother with my little “sneaky feeling”, ya know?
All in all, I just think @mattcutts wants folks to know that one human rater cannot push a button and pop your site. I’m not saying NO ONE can do that, I’m just applying that statement to the abilities of the work-from-home outsourced Quality Raters.
Knowing that the outsourced Raters don’t have that ability IS good to know. That is a terrifying thought after reading all through their forum.
However, what ELSE happens to that data is another story… and something we probably will never know the exact details about.
Jennifer
“Matt didn’t have to stop by and say ANYTHING. ”
What do you think Matt’s REAL job is? It’s not fighting webspam, it’s spinning for Google.
He isn’t doing you or me any favors, he is doing what he thinks it’s best for the company that pays him all that money. Never forget that, he lies and spins for money.
Jennifer-
It only stands to reason from another angle as well: Google is in the BUSINESS of AUTOMATING SEARCH RESULTS.
Sorry about the extra emphasis, but I think it drives the point home. Google’s business is about improving automation of results not manually investigating the BILLIONS of PAGE 1s for major search terms.
I mean, can you imagine the army of reviewers Google would need to review JUST PAGE 1 of search results for even just the two and three keyword phrases? They’d be out of business in a month from the labor expense (and optical insurance from all the dreary eyes).
Read any kind of article about the “business environment” within Google and you’d KNOW dreary eyes are NOT supposed to be part of the work experience.
I completely agree. I’ve had an ongoing argument with my business partner who is convinced that raters will be looking at all of our sites. AS IF Google has the labor to check out the top ranking sites for all the billions (if not trillions) of possible keyword phrases!
No one ever said they even come close to rating ALL query spaces. That is impossible – especially since 15-20% of all queries each month are NEW phrases that have never been used before.
They (raters) evaluate small samples then that small dataset is reviewed. If they (the algo team decides to move forward with testing the algo tweak then feeds out to a small sample of live Google traffic. LOTS happens before it becomes an official change in the algo.
Jennifer
Very interesting post that has left me with several new questions about how Google works. Unfortunately, these are questions that I’ll never get answers from Google on.
Most likely no one will =)
Jennifer
Not sure why everybody is hating on Matt Cutts here. He doesn’t have to give out any information at all. He could sit back, be a ghost, and never say a word to the public, but he doesn’t do that. He has made hundreds of webmaster videos, has a very insightful blog, and takes the time out of his day to comment on blogs such as this one.
Does he give out all the information he knows about search? Of course not, that would be foolish. There is already enough black haters out there spamming the internet without Matt giving away every single secret about search.
I don’t see it as people hating on Matt Cutts. Seems to me he provided a lot of clarity, a good thing IMO.
This is a great discussion. It’s good to have things cleared up by Matt.
I would also like to speculate that Matt Cutts may be under some kind of NDA agreement. So, if there are times when he is a little “cagey”, it’s could be because he can’t talk without getting into big trouble with the boss.
This doesn’t mean that he will outright lie about what’s going on inside Google.
Matt strikes me as super well-intentioned, transparent, and honest. But quite a few times his assertions were debunked by Google’s actions. I dont know, but it seems logical they’d take a site that has been poorly rated by 70-80% of raters into the hands of somebody in the webspam team.
The way he’s explaining it makes a lot more sense – it’s much more scientific to test their algo tweaks offered up by the webspam team through quality control teams,it’s basic scientific method.
There are legitimate questions about the future of affiliate marketing or other activities done in search, but I don’t see a reason to question Matt’s integrity as a person: he may dodge a question, but I’ve never seen him in particular come out and lie about something so plainly.
(I could be wrong of course, but not that I’ve yet to see.)
I have never really thought about being marked as spam in any way. Shouldn’t our content be of quality, not only to avoid this, but to get ranked higher?
So quality raters affect the measuring stick, which indirectly affects not just one site, but all sites.
Good to hear. Matt is correct about his “Rule #1”. It’s good Engineering practice. Maintain a focus on evaluation of the outcome of the algo test, and don’t mix that observation data with any filtering or other changes to the sample being used to evaluate the algo.
I don’t buy the idea that internally, no one at Google is ever influenced by that data, however. It’s too big a honey pot *if the data is good*.
As we’ve seen time and again with large companies who save data, eventually someone attempts to climb the career ladder by getting “innovative” with “latent assets” like that sort of data. It’s good to hear the current crop of Googlers are disciplined enough to follow the lead of their Engineering teams.
I’m more inclined to believe that quality rater data is NOT clean, quality data. I suspect that few care about the quality of it beyond it’s utility as an evaluation of the quality of the algo changes, especially people in Matt’s group. They would be threatened by good, clean crowd sourced data like that, unless it was typical for loosely managed crowd sourced data — unreliable for much more than gross aggregate metrics.
Let’s analyze what Google’s public relations face has said: human raters paid a decent wage by Google see your site, mark it as spam several times and nothing happens to it. Yeah, and cigarette smoking is not addictive, otherwise the cigarette companies would not have said it. Corporations have no reason to tell you the truth, especially when the truth interferes with their tens of billions in profits.
If Google admits to playing such a huge role in results they are exposed to potential legal liability, given that it is not algorithmic. Matt has lot of reasons to play with words, mislead or outright lie. If he doesn’t do it, Google hires another public relations person. (You should have read a thread when he tried to deflect from the fact that Google was making so much money after Panda.)
Is Matt still on the spam team? It appears to me that he moved to the spin team a year or two ago, just as Google started to truly drive small sites out of business. Always question things and never say “but he /she looks like a nice guy so the company must be nice.”
Regardless, I’m still glad Matt stopped by. Man, what that guy opens himself up to each time he opens his mouth…like these kinds of comments I’ve seen here. This is his job. At least he tries to answer questions, but he still has to protect the company, ya know?
Not everyone listening to him has GOOD intentions – and that, by itself, is good enough reason to chose your words carefully.
Just my 2 cents.
Jennifer
I think that it’s impossible to manually rate some urls as there are millions of new posts and websites created each and every day and it would be impossible to rate even some of them. And it wouldn’t be fair games as those manual raters has their own subjective opinion so the internet business opportunities won’t be equal for everyone because not many sites gets manually reviewed.
At least we know Matt is listening. For a while I thought he was lost in his own algorithm per say and not really paying attention to what is really going on.
Like many folks here, I have had a handful of my quality sites get banged around Google due to the Panda algo and “testing” of algo’s. I get it, but as of right now the search results seem dated (like 2002-2006) when searching for obviously new stuff.
There is a difference between “selling” and “spamming” as everyone has their own opinion about it. But it’s Google’s opinion that matters most, right?
Its really hard to believe me that Google does not take in account the ratings and that too of quality raters.
Does this exist now too?
Just goes to show that sometimes it’s a good idea to speculate – maybe we need to do a bit more of that and provoke Google into being more transparent about their algorithm quality control.
Seems that quality raters have equity to effect ranking in the future. Not at the time of flagging but potentially they could be the catalyst to change future rankings certainly.
PotPie — Thanks for bringing up the topic and raising it to a level where G responded. I admit I’m still a bit confused by the role of the quality raters, but at least I know more now than I did a couple days ago.
As nice as Matt Cutts is, do people seriously think they can rely on the head of web spam to tell them key facts or truths that could undo his web spam works. I know an SEO from online with a decent level of respect in the field of SEO and he reckons he pulled up a Google employee for lying… And the employee admitted lying to him lol. #JustSayin 😉
Got Matt Cutts attention? Nice work! I think Cutts is an genuine, honest guy. I don’t think is going to blatantly give away the algorithm, but helpful nonetheless.