Netconcepts

Specialists in SEO, web dev, online marketing

Book A Free Strategy Call
  • About
    • Problems We Solve
    • Who We’re Not a fit for
  • Services
  • Results
  • Learn
    • Learning Center
    • Courses
    • Blog
    • Podcasts
  • Press
  • Contact

Good Cloaking, Evil Cloaking & Detection

Is cloaking evil? It’s one of the most heavily debated topics in the SEO industry – and people often can’t even agree on what defines cloaking. In this column, I wanted to look at an example of what even the search engines might consider “good” cloaking, the middle-ground territory that page testing introduces plus revisiting how to detect when “evil” old-school page cloaking is happening.

Back in December 2005, the four major engines went on record at Search Engine Strategies Chicago to define the line between cloaking for good and for evil. From the audience, I asked the panelists if it was acceptable to – selectively for spiders – replace search engine unfriendly links (such as those with session IDs and superfluous parameters) with search engine friendly versions. All four panelists responded “No problem.” Charles Martin from Google even jumped in again with an enthusiastic, “Please do that!”

URL Rewriting? Not Cloaking!

My understanding is that their positions haven’t changed on this. Cloaking – by its standard definition of serving up different content to your users than to the search engines – is naughty and should be avoided. Cloaking where all you’re doing is cleaning up spider-unfriendly URLs, well that’s A-OK. In fact, Google engineers have told me in individual conversations that they don’t even consider it to be cloaking.

Because search engines are happy to have you simplify your URLs for their spiders – eliminating session IDs, user IDs, superfluous flags, stop characters and so on – it may make sense to do that only for spiders and not for humans. That could be because rewriting the URLs for everyone is too difficult, costly or time intensive to implement. Or more likely, it could be that certain functionality requires these parameters, but that functionality is not of any use to a search engine spider – such as putting stuff in your shopping cart or wish list or keeping track of your click path in order to customize the breadcrumb navigation.

Many web marketers like to track which link was clicked on when there are multiple links to the same location contained on the page. They add tracking tags to the URL, like “source=topnav” or “source=sidebar.” The problem with that is it creates duplicate pages for the search engine spiders to explore and index. This leads to a dilution of link gain or PageRank, because all the votes that you are passing on to that page are being split up because of the different URLs you are using. Ouch.

How about instead you employ “good cloaking” and strip out those tracking codes solely for spiders? Sounds like a good plan to me. Keep your analytics-obsessed web marketers happy, and the search engines too.

I have to mention, you don’t have to cloak your pages to simplify your URLs for spiders. There is another option: you could use JavaScript to append your various tracking parameters to the URL upon the click. For example, REI.com used to append a “vcat=” parameter on all brand links on their Shop By Brand page through JavaScript. Thus, none of their vcat containing URLs made it into Google.

Is Testing Bad Cloaking?

Is multivariate testing a form of bad cloaking? This is where services like Offermatica or even Google’s own Website Optimizer show different users different versions of the same URL. That could be considered cloaking, because human visitors and search engines are getting different content. Spiders can’t participate in the test group, and thus the content of that test is invisible to the spiders; that’s because of the requirements of AJAX, JavaScript, DHTML and/or cookies for the test platform to function on the user’s browser. Google engineers have told me that they want Googlebot to be part of the test set. Therein lies the rub; the technology isn’t built to support that.

Uncovering User Agent Based Cloaking

The “bad” cloaking from a search engine point of view is that deliberate showing to a spider content that might be entirely different than what humans see. Those doing this often try to cover their tracks by making it difficult to examine the version meant only for spiders. They do this with a “noarchive” command embedded within the meta tags. Googlebot and other major spiders will obey that directive and not archive the page, which then causes the “Cached” link in that page’s search listing to disappear.

So getting a view behind the curtain to see what is being served to the spider can be a bit tricky. If the type of cloaking is solely user agent based, you can use the User Agent Switcher extension for Firefox. Just create a user-agent of:

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

under Tools > User Agent Switcher > Options > Options > User Agents in the menu. Then switch to that user agent and have fun surfing as Googlebot in disguise.

Uncovering IP Based Cloaking

But hard-core cloakers are too clever for this trick. They’ll feed content to a spider based on known IP addresses. Unless you’re within a search engine — using one of these known IP addresses — you can’t see the cloaked page, if it also has been hidden by being kept out of the search engine’s cache.

Actually, there’s still a chance. Sometimes Google Translate can be used to view the cloaked content, because many cloakers don’t bother to differentiate between the spider coming in for the purpose of translating or coming in for the purpose of crawling. Either way, it uses the same range of Google IP addresses. Thus, when a cloaker is doing IP delivery they tend to serve up the Googlebot-only version of the page to the Translate tool. This loophole can be plugged, but many cloakers miss this.

And I bet you didn’t know that you can actually set the Translation language to English even if the source document is in English! You simply set it in the URL, like so:

http://translate.google.com/translate?hl=en&sl=en
&u=URLGOESHERE&sa=X&oi=translate&resnum=9&ct=result

In the code above, replace the bolded URLGOESHERE part with the actual URL of the page you want to view. That way, when you are reviewing someone’s cloaked page, you can see the page in English instead of having to see the page in a foreign language. You can also sometimes use this trick to view paid content, if you’re too cheap to pay for a subscription.

Many SEOs dismiss cloaking out-of-hand as an evil tactic, but in my mind, there is a time and a place for it (the URL simplifying variety, not the content differing variety), even if you are a pearly white hat SEO.

deca steroid

Comments

  • BusinessCardGuru says:
    May 1, 2007 at 6:02 am

    Wow… A great article. I will definitely try the detection of bad cloakers.

    Reply
  • Postma SEO Blog says:
    March 6, 2008 at 1:44 pm

    Thanks for this article, will try to write a review on my blog …

    Reply
  • The Art of SEO

    Chapter 7:
    Content Marketing

    From the fundamentals of link building to the nuances of natural linking patterns, virality, and authority.

    Get Free Chapter

    Related Posts

    Blog Images

    Your Ultimate Guide to Google Penalties

    When your website’s traffic suddenly takes a tumble and you don’t know why, it can be maddening. Why is this happening? It’s possible you’ve been hit with a Google penalty, or have come up against one of Google’s algorithm updates. If you’re wondering how Google penalties work and what you should do to avoid them, […]

    Read More
    Blog Images

    Using Psychology to Boost Your Conversion Rate Optimization

    Can a better understanding of the human brain help you get more conversions? Absolutely! Let’s look at how to incorporate some psychological concepts into your conversion rate strategy, along with some examples. Behavioral economics: humans are irrational What is behavioral economics and what does it have to do with conversion rates? Traditionally, economists believed that […]

    Read More
    Blog Images

    How to Master Social Selling on LinkedIn

    Social selling could be your key to securing new prospects and boosting your bottom line. When not done properly, however, social selling can also ruin your online presence and repel potential customers. So what is social selling, and why does your brand need it? More importantly, how do you master social selling? Let’s look at […]

    Read More
    As seen on
    NBC ABC CBS Fox CW Haryard adweek foundr cnet
    Capsule Bar Img
    Will your content marketing turn readers into customers?

    Download chapter 7 of The Art of SEO, Content Marketing

    Download Now
    netconcepts
    • submenu
      • About
      • Services
      • Problems We Solve
    • submenu
      • FAQ
      • Results
      • Who We’re Not a fit for
    • submenu
      • Case Studies
      • Testimonials
      • Learning Center
    • submenu
      • Courses
      • Blog
      • Podcasts
    • submenu
      • Press
      • Contact
    netconcepts
    © 2021-2022 Netconcepts. All Rights Reserved. Terms of Use Earnings Disclaimer Privacy Policy
    Book

    The Art of SEO

    Chapter 7: Content Marketing

    Yes, this is the book that Seth Godin referred to as “Hype-free, data-rich, and loaded with insight that’s essential reading for anyone who needs a deep understanding of SEO.”

    Want to sharpen your SEO chops?...
    start out with an appetizer before you dig into the main course!

    Get Free Chapter
    • Facebook
    • LinkedIn
    • Pinterest
    • Twitter
    • YouTube