Meet the Enemy: Your Most Ardent Users (and what to do about it)
Posted by Elf Sternberg as Uncategorized
The title is tongue-in-cheek, but take it in good faith: your most ardent users, the ones who link to you and provide you with both traffic and search engine legitimacy, can also be your biggest nightmares. Because when they link to you, they won’t just link to your home page. They’ll link to what they love deep within your site, and six months later, when you re-arrange, those links will go to odd places and crash zones, and GoogleBot and BingBot will follow those links and fill your error logs.
Dealing with those is a chore, but here’s what I’ve recently discovered: filter them out. Filter out all hits from GoogleBot and BingBot from your error logs and you’ll be left with a gem: the errors caused by human beings actually using your site. Target those first, then work on closing the nuisances caused by automated probes.