Spider trap for pages created on the fly
Q: Please explain again the “spider trap” as it pertains to pages created on the fly. How big of a disadvantage does this present? If redesigning the site isnt an option, what can be done to lessen this disadvantage?
A: A search engine spider can get caught in a “spider trap” if it keeps bumping into links to pages that are the same content but with different URLs that are varied dynamically (e.g. where the URLs contain a nonessential variable/flag or session ID in the query string). If caught in a spider trap, the spider would download the same pages over and over again, overloading the site’s web server and cluttering up Google’s index with a slew of duplicates. To circumvent such potential problems, Googlebot often chooses to skip over various dynamic pages. This can have very deleterious results, such as the majority of a dynamic site getting skipped over by Googlebot. If revamping the site isn’t an option, you might want to consider an outsourced “dynamic feed” service such as GravityStream.
Chapter 6:
Keyword Research
From the fundamentals of link building to the nuances of natural linking patterns, virality, and authority.
Related Posts
Embrace Journaling, Tackle Tardiness, and Explore Our Energetic Echo
Here’s what I found inspirational, challenging, or just downright hilarious this week. What caught your eye? And, remember to check out this week’s great podcast episodes: Scaling a SaaS Company with Jason Morehouse “A crucial factor to business success is to find and take the personal path that works best for you.” — Jason Morehouse […]
Read More
Harrison’s harmony, conquering a blank canvas, & gut health hacks
Here’s what I found inspirational, challenging, or just downright hilarious this week. What caught your eye? And, remember to check out this week’s great podcast episodes: Be a Sales Game Changer with Fred Diamond “True elite sales professionals develop a dedicated mindset, proactive client interaction, and continuous self-preparation. They understand their client’s needs and enable […]
Read More
Rebirth of sleeper trains, 4,000 weeks is a long/short time, and golden age for medicine
Here’s what I found inspirational, challenging, or just downright hilarious this week. What caught your eye? And, remember to check out this week’s great podcast episodes: A Story Worth Retelling with Luke Storey “Aligned values are the cornerstone of successful partnerships, whether in business or life, as they shape our moral code, define our priorities, […]
Read More
