Do no evil? The Google & JC Penney controversy

We usually associate black hat Search Engine Optimisation (SEO) tactics with spammy sites selling get rich quick guides, or porn, or any other slightly dubious online business. This is why it is so surprising (and intriguing) when a major US retailer gets caught with its SEO pants down.

The New York Times has released an article describing how black hat SEO tactics allowed American retailer JC Penney to rank first on Google’s Search Engine Results Pages (SERPs) for a large number of keywords.

JC Penney, which has very successful offline and online business, was ranking in the top spot across a variety of different product areas, from dresses to bedding, beating out arguably more relevant competitors. In one particular case, JC Penney was ranking first for the search term ‘Samsonite carry on luggage’, ahead of Samsonite’s own website. JC Penney also happens to be one of Google’s biggest buyers of paid search advertising.

A quick lesson in SEO

To understand how JC Penney’s website was achieving these results, it is important to understand one aspect of how Google (and other search engines) rank websites. Search engines judge the importance of a website in part by the amount of links coming into it, and by the quality of those links. So, if your site has three incoming links, and your competitor’s has 3000, then all other things being equal, your competitor’s site will rank higher on Google for the same search term.

If you both have only one incoming link, and the link to your site is from The New York Times, and the link to your competitor’s site is from Jimmy’s Plumbing Services, then all things being equal you will rank higher. So both amount and quality count.

A quick lesson in Black Hat tactics

Google and other search engines use this measure of importance because mostly, it works. It makes sense. Of course once people understood this aspect of Google’s ranking algorithm , they try to play the system. Black Hat methods are those that are frowned upon by search engines because they help a website give the impression of relevance and importance, whilst actually it might be neither of the two. One basic example would be if you paid 3000 website owners to put links on their sites, pointing to your site, no matter what their sites were about. So right in the middle of a site about Star Wars costumes, one might find a link to Jimmy’s Plumbing Services, for example. For Google to use this link as a sign that a website is just that little bit more important would be an example of Google being tricked by a Black Hat SEO method.

What did JC Penney do?

JC Penney claims it did nothing, and was completely unaware that these methods were being used to improve its search rankings. Upon hearing the news, it promptly fired its search agency, SearchDex. What we do know is that thousands of links were placed on thousands of irrelevant websites, all pointing towards JC Penney.  The websites that the links appeared on mostly use companies such as TNX.net, who pay website publishers to put the links on their site. TNX get paid by clients such as JC Penney (or someone on their behalf) to do this and the result is Google and other search engines mistakenly calculating a large amount of meaningful and relevant links pointing towards the clients’ websites, and pushing them up in the SERPs as a result.

Why did Google not find out?

This is the big question – Google had previously ‘punished’ JC Penney for other Black Hat SEO techniques, yet failed to pick up a relatively large black hat campaign that was keeping JC Penney in the number one spot on numerous highly valuable keywords for a four month stretch. Google’s official line, from resident king of anti-spam Matt Cutts is that Google serves one billion searches a day and cannot possibly pick up all questionable SEO tactics. According to Cutts, Google does extremely well at dealing with spam, considering the vast scales of their system, and are constantly evolving in a battle he does not think will ever end.

So what is the controversy?

What adds an element of conspiracy to the story is that as mentioned earlier, JC Penney pays Google considerable sums of money for paid search advertising through AdWords. To put a number on it, US$2.46-million a month according to an internal Google document discovered by Advertising Age. The implication is clear – does Google turn a blind eye to black hat SEO methods used by its largest customers?

Of course it is a possibility, but currently Google is not in the position of having to fight for customers. It is the de facto search engine of choice for the majority of web users, and companies such as JC Penney probably do not have to be encouraged to spend their advertising budgets there in the way that this conspiracy theory suggests. What is perhaps more likely, is that whilst JC Penney got caught with its pants down, so did Google for not catching them out.

What is also interesting to consider is the game theory element brought to light by this revelation. The economic incentive for businesses to rank highly on search engines is now undeniably important. Ranking number one for popular keywords is significantly more lucrative that ranking at number two, and so on. This means that if there are black hat methods available to companies, or to search agencies acting on their behalf, it is very much in their interests to use them, as long as they do not get caught.

Once one major player starts, eventually all others have to follow suit if they are not to fall behind. The implications of this are that if JC Penney (or it’s agency) has been caught doing this, it’s a fair bet that lots of other major companies are doing it too – especially in the online retail industry. With all eyes on Google, and antitrust watchdogs salivating with anticipation, it’s going to be both exciting and worrying to see if any other big names get dragged into the spotlight.