Hey, Matt Cutts, I’m using JavaScript to hide links from Google, cool?

by SEO Mofo on Jun 16th, 2010

in SEO Ethics

Recently, I published an article called Advanced SEO for Affiliate Marketing Links, which explains linking techniques that prevent the transfer of PageRank to affiliate links, while also allowing the page to load faster–in some cases, a lot faster. However, the article has been generating some concerns amongst my fellow SEO practitioners that perhaps my advanced SEO linking techniques are actually violating Google’s Webmaster Guidelines.

UPDATE: I submitted a question to the Google Help Forums, asking whether or not this technique violates the Google Webmaster Guidelines. There were quite a few answers from the “Top Contributors” and even a Google employee responded. As I expected, their responses focused on the fact that this technique manipulates Google, and they desperately tried to avoid the inescapable reality that this method improves the User experience. So the question basically came down to this: is it okay to hide content from Google if it results in a better user experience? The jury is still out, but I’ll update again when I get an answer. Meanwhile, you can follow the heated debate on the Help Forums: Can I use JavaScript to hide affiliate links from Google?

I’d like to take a moment to share my thoughts on this issue, but I’d also like to hear some outside opinions from other SEO/SEM folks–especially all you white hat extremists out there. Hopefully I can persuade Matt Cutts or someone from the search quality team to comment on this post and provide a more definitive answer. Personally, I’m very interested in Google’s official stance on this issue, since…um…I’m currently using these techniques all throughout this site, and if I get booted from Google’s index…my nine children will surely starve.

What are these advanced SEO techniques you speak of?

I should probably recap the techniques I wrote about, to make sure everyone understands what we’re talking about here. Here’s a quick overview:

  1. Remove affiliate links from your pages’ HTML code.
  2. If it’s a text link embedded in page content, then just remove the <a> tags and leave the anchor text there (as plain text).
  3. If it’s an image link, remove the <a> tags AND the <img> tag.
  4. Replace the <a> tags with <span> tags, and assign a class name to them (e.g. <span class="affiliate">).
  5. Write a JavaScript function that looks for <span> elements that contain class="affiliate" and replaces them with your affiliate links. (Yes, the ones you removed in Step 1).
  6. Put the JavaScript function in an external .js file and block Google from accessing it (using robots.txt).
  7. Attach the JavaScript function to the onload event, which means users’ web browsers won’t call the function until after the page is finished loading.
  8. Googlebot–and users without JavaScript enabled–can’t call the JavaScript function, so your affiliate links are never inserted into the page content.
  9. Only users with JavaScript-enabled browsers will see your affiliate links. Everything/everyone else sees plain text instead of affiliate text links, and an empty space instead of affiliate image links.

If you still don’t get it, check out this awesome diagram I spent 6 hours making just for you: lazy-loading affiliate marketing anchors.

If you still don’t get it, even after viewing the diagram, then there’s no hope for you. I don’t mean just with this article…I mean your entire life is hopeless and you should seriously consider killing yourself. However, if you do understand…then let’s continue.

The Big Question: Does the practice of lazy-loading affiliate marketing links violate the Google Webmaster Guidelines?

I’ll kick off this debate by quoting some of the related information from Google’s Webmaster Guidelines–specifically the Quality Guidelines portion, which focuses on the things that might get your site penalized.

To make things more entertaining, I’ll turn this into an argument between 2 imaginary characters, whom I’ve arbitrarily named Blackie and Whitey. In this debate, Whitey thinks the SEO technique in question is deceptive and shouldn’t be used. Blackie thinks the SEO technique in question is not inherently evil whatsoever and it can actually benefit users, webmasters, AND search engines. Read what these guys have to say…and then feel free to express your opinion in the comments section.

Webmasters who spend their energies upholding the spirit of the basic principles will provide a much better user experience and subsequently enjoy better ranking than those who spend their time looking for loopholes they can exploit.

Blackie: The SEO technique in question is not designed to exploit any loopholes; it’s designed to benefit users, Google, and me…the webmaster. Users benefit from faster page load times. Google benefits because it doesn’t have to waste resources trying to crawl affiliate links, and because it avoids contaminating the link graph with links that have no inherent editorial endorsement from the affiliate marketer. And as a webmaster, I benefit from higher rankings that might result from minimizing “PageRank evaporation” and decreasing page load times.

Whitey: The SEO technique in question is clearly exploiting loopholes. The most obvious example is the fact that it’s not really decreasing the amount of time it takes to load a page. All it’s doing is delaying the loading of affiliate links/content until after the main content is loaded. If we ignore the onload event, the amount of time it takes to load ALL of the content has actually INCREASED, because the user’s web browser now has to request and process the external JavaScript file.

Blackie: First of all, webmasters can add the code to their existing .js files, so this technique doesn’t necessarily require an extra HTTP request.

Second of all, what the hell are you talking about, “if we ignore the onload event”? While we’re at it, let’s ignore the definition of a “second,” and I’ll argue that this SEO technique reduces page load times by 2 millennia.

The fact is…the onload event is extremely important and cannot be ignored. It’s the web browser’s way of saying “Ok, User, this web page is done loading, and you are now free to begin perusing the content.” It’s also the point at which the “Loading…” message/icon goes away. Sure, some affiliate banners might appear a few seconds later, but that has little or no effect on the user’s experience. All the user cares about is accessing the main content as quickly as possible, and with this advanced SEO technique…the user doesn’t have to wait for your affiliate banners to load.

Whitey: Whatever. Let’s move on to the next section and see if your SEO trick upholds “the spirit of the basic principles.”

Make pages primarily for users, not for search engines. Don’t deceive your users or present different content to search engines than you display to users, which is commonly referred to as “cloaking.”

Whitey: There you go, Blackie! Read it and weep! It clearly says “DO NOT present different content to search engines than you display to users.” Sorry, but that’s exactly what you’re doing. You’re cloaking. End of story.

Blackie: No…NOT “end of story.” Cloaking uses server-side programming to serve different content to different user-agents or IP addresses. My server returns the exact same document to users and search engines. If a user views the HTML source of my page, they will see exactly what Google sees. The HTML code is identical, down to every last character.

Whitey: Yes, but a web page is much more than just HTML code; it’s also CSS, JavaScript, images, Flash objects, videos, and all kinds of other stuff. You may be serving the same HTML document, but you’re not allowing Googlebot to fetch your JavaScript file, and it is this file that instructs web browsers to rewrite the HTML code internally to show affiliate links and ads. So ultimately, the user sees their browser’s rewritten HTML, which is different from the HTML Googlebot sees.

Blackie: First of all, I highly doubt that Google even requests external files to begin with. Even if it did, I don’t imagine Google is capable of parsing JavaScript, using it to rewrite the HTML of any documents that referenced it, and then retroactively updating those documents’ index data to reflect the newly-discovered content. So unless Google is actually trying to fetch my JavaScript–which it isn’t–then blocking it with robots.txt isn’t really doing anything anyway.

Second of all, bandwidth costs money, and I’m not obligated to serve Googlebot anything I don’t want to. It’s my JavaScript file; I have the right to control access to it.

Whitey: You’re right…but similarly, it’s Google’s index and Google has the right to ban your website from it.

Blackie: Yeah yeah yeah…blow me. Next quote.

Avoid tricks intended to improve search engine rankings. A good rule of thumb is whether you’d feel comfortable explaining what you’ve done to a website that competes with you. Another useful test is to ask, “Does this help my users? Would I do this if search engines didn’t exist?”

Whitey: The SEO technique in question is definitely a trick intended to improve search engine rankings.

Blackie: True, the technique is intended to improve search engine rankings, but it’s not a “trick.” That’s like saying AJAX is a trick. Or Flash is a trick. There are always going to be emerging web technologies and languages that Google can’t crawl or doesn’t understand. That doesn’t mean using them is intended to trick search engines.

Plus, let’s consider the two suggested ways of testing whether or not this technique should be used. “Does this help my users?” YES. “Would I do this if search engines didn’t exist?” YES.

Whitey: But would you feel comfortable explaining what you’ve done to a website that competes with you? I doubt it.

Blackie: Um…I spent the last three days trying to explain this, in vivid detail, to every SEO/SEM “competitor” who will listen. I even made a diagram, so idiots like you would understand it.

Whitey: Yeah yeah yeah…suck it. Next quote.

Use of JavaScript is an entirely legitimate web practice. However, use of JavaScript with the intent to deceive search engines is not. For instance, placing different text in JavaScript than in a noscript tag violates our webmaster guidelines because it displays different content for users (who see the JavaScript-based text) than for search engines (which see the noscript-based text).

Whitey: Gee…that sounds kinda like what I was saying.

Blackie: The SEO technique in question doesn’t deceive anyone. I’m explicitly telling search engines they can’t request my JavaScript file, and therefore they can’t see the content it generates. I’m not displaying different content to search engines; I’m simply telling them this content is for humans only…no bots allowed. I mean come on, Whitey, surely you of all people can understand the value of segregation and discrimination.

Whitey: Of course. Wait…what?!

Blackie: Next quote, please.

Avoid hidden text or hidden links.

Whitey: Obviously, your technique is a way to hide links from search engines.

Blackie: I also disallow search engines from trying to crawl my contact form. Does that mean I’m hiding my email address from Google? And I also disallow search engines from crawling my infinite calendar pages. Does that mean I’m hiding my dentist appointment next week?

I’m using JavaScript to create links that only users can see; there’s nothing in the Quality Guidelines that says you must show Google everything that users see. In many cases, that’s not even possible. The hidden links section of the Guidelines even defines hidden links as “links that are intended to be crawled by Googlebot, but are unreadable to humans.” That’s not what I’m doing. I’m doing the exact opposite of that.

Whitey: I just realized something…you’re right. This SEO technique is awesome. Where can I subscribe to your blog?

Blackie: You can click the RSS icon below. It’s one of the 18 social media buttons I lazy-loaded onto this page while you were reading the article content. I didn’t want to make you wait.

Whitey: Well thank you. I appreciate that. Your lazy-loading trickery has indeed improved my user experience here today. I was a fool for doubting you, Blackie. Thankfully, the fog has been lifted, and for the first time…I can clearly see the–

Blackie: –Just click the icons.