Spider trap for pages created on the fly
Q: Please explain again the “spider trap” as it pertains to pages created on the fly. How big of a disadvantage does this present? If redesigning the site isnt an option, what can be done to lessen this disadvantage?
A: A search engine spider can get caught in a “spider trap” if it keeps bumping into links to pages that are the same content but with different URLs that are varied dynamically (e.g. where the URLs contain a nonessential variable/flag or session ID in the query string). If caught in a spider trap, the spider would download the same pages over and over again, overloading the site’s web server and cluttering up Google’s index with a slew of duplicates. To circumvent such potential problems, Googlebot often chooses to skip over various dynamic pages. This can have very deleterious results, such as the majority of a dynamic site getting skipped over by Googlebot. If revamping the site isn’t an option, you might want to consider an outsourced “dynamic feed” service such as GravityStream.
Comments
Chapter 7:
Content Marketing
From the fundamentals of link building to the nuances of natural linking patterns, virality, and authority.
Related Posts
Your Ultimate Guide to Google Penalties
When your website’s traffic suddenly takes a tumble and you don’t know why, it can be maddening. Why is this happening? It’s possible you’ve been hit with a Google penalty, or have come up against one of Google’s algorithm updates. If you’re wondering how Google penalties work and what you should do to avoid them, […]
Read More
Using Psychology to Boost Your Conversion Rate Optimization
Can a better understanding of the human brain help you get more conversions? Absolutely! Let’s look at how to incorporate some psychological concepts into your conversion rate strategy, along with some examples. Behavioral economics: humans are irrational What is behavioral economics and what does it have to do with conversion rates? Traditionally, economists believed that […]
Read More
How to Master Social Selling on LinkedIn
Social selling could be your key to securing new prospects and boosting your bottom line. When not done properly, however, social selling can also ruin your online presence and repel potential customers. So what is social selling, and why does your brand need it? More importantly, how do you master social selling? Let’s look at […]
Read More
