u can add autohitsnow.com to the list, they got one of mine banned at fab.
If you are new to domains and looking to buy, sell and learn about domains then you have come to the right place. DNForum is the largest domain name community on the internet and continues to grow every day. There are over 105,000 domainers on DNForum doing everything from buying domains, selling domains, learning about domains and discussing domains. Take a minute and Register.
Register Today on DNForum IT'S FREE!in light of this zero tolerance policy on dodgy referers.
Here is a little PHP snippet to filter them out before you send your traffic.
This will filter out traffic coming from:PHP Code:<?
$ref = $_SERVER["HTTP_REFERER"];
if ( false !== strpos( $ref, "webwidesurf" ) )
{
header("Location: http://www.articledirectory.net/");
exit;
}
else if ( false !== strpos( $ref, "autosurfpro" ) )
{
header("Location: http://www.articledirectory.net/");
exit;
}
else if ( false !== strpos( $ref, "globalblaster" ) )
{
header("Location: http://www.articledirectory.net/");
exit;
}
else if ( false !== strpos( $ref, "1millionvisitors" ) )
{
header("Location: http://www.articledirectory.net/");
exit;
}
else
{
header("Location: http://www.searchnut.com/?domain=domain.com");
exit;
}
?>
1millionvisitors.com
globalblaster.com
autosurfpro.com
webwidesurf.com
Enterprise Communications and Social Media in Uist
u can add autohitsnow.com to the list, they got one of mine banned at fab.
many thanks.. added that now
also just designed a bot catcher..
of course you might not want to redirect GOOGLE or some of the other search engines for obvious reasons.PHP Code:$spiderList =
array(
'googlebot'=>'GOOGLE',
'msnbot'=>'MSN',
'VerticalCrawler'=>'YAHOO via Overture',
'scrubby'=>'Scrub The Web',
'exactseek'=>'ExactSeek',
'linksmanager'=>'LinksManager.com',
'infoseek'=>'Infoseek',
'jeeves'=>'Direct Hit/Teoma/Ask Jeeves',
'robozilla'=>'Netscape Directory / DMOZ Open Directory',
'scooter'=>'Altavista robot',
'slurp'=>'Inktomi YAHOO',
'speedy'=>'Entireweb.com',
'W3C_Validator'=>'W3C HTML-Code Validator',
'MantraAgent'=>'Looksmart',
'overture'=>'Overture/Fast/Alltheweb',
'url fetching'=>'PERL SCRIPT PAGE FETCHER',
'almaden'=>'IBM Crawler',
'Snapbot'=>'Snapbot 1.0',
'ScSpider'=>'ScSpider 0.2',
'ia_archiver'=>'Alexa Crawler',
'xabot-Images'=>'Exalead image crawler',
'BaiduSpider'=>'Baidu Crawler',
'Exabot'=>'Exalead Search Crawler',
'EmeraldShield'=>'EmeraldShield Bot',
'EMPAS_ROBOT'=>'Empas Korea Crawler',
'lanshanbot'=>'lanshanbot Chinese Bot',
'Mediapartners-Google'=>'Google Adsense Bot',
'voyager'=>'Kosmix Crawler',
'Yahoo-MMCrawler'=>'Yahoo Image Crawler',
'libwww'=>'libwww crawler',
'SBIder'=>'SiteSell.com',
'sogou spider'=>'Fake Chinese sogou spider',
'linkwalker'=>'linkwalker seventwentyfour.com',
'NaverBot'=>'NHN Corp www.naver.com',
'Psbot'=>'Picsearch Image Crawler',
'aipbot'=>'NameProtect Copyright Search Bot',
'MVAClient'=>'Unkown MVAClient',
'ichiro'=>'ichiro - japanese crawler',
'Syntryx' =>'Syntryx Solution Suite',
'ShopWiki'=>'ShopWiki',
'SurveyBot'=>'SurveyBot whois.sc',
'WISEbot'=>'WISEbot Korea',
'192.comAgent'=>'192.comAgent',
'Nusearch'=>'Nusearch Spider',
'BecomeBot'=>'Become.com spider',
'InternetSeer'=>'InternetSeer');
foreach ($spiderList as $agent => $desc)
{
if(stristr($useragent,$agent)==TRUE)
{
header('HTTP/1.1 200 OK');
header("Location: http://www.domain.com/"); //Redirect the bots
exit;
}
}
However if you want a true representation of what traffic you send to a particular program then filter them all.
Alternatively you can just record them into a database and filter them out from your stats.
in August I will be posting the results of this test.. Giving full details of which programs DISPLAY ACTUAL statistics and which are the less honest ones, obviously we have included a small percentage for a margin of error caused by failed requests, but we record EVERY single visitor we refer to the respective programs.
We are logging the top 5 programs at the moment.
Last edited by namestrands; 06-17-2006 at 10:03 AM.
Enterprise Communications and Social Media in Uist
Real search engine spiders aren't normally bad, because they won't hammer us or click out. But that's my 2 cents. But pay per view your site or paying people to click on your site will immediately get you banned.
Donny
Donny if you have a list, I will develop a script for the domaining public to enable them to stop this, as more often than not it is not the fault of the actual domainer. This would give the more honest domainers a chance to help improve domain traffic quality.
The filtering out of Bots is more of a stats analysis excercise, and I would encourage people to filter out the crap bots and record the crawler bots so they can have more transparency. Which will allow them to compare their stats with the stats of the parking programs
Enterprise Communications and Social Media in Uist
Say someone came across a PPC site and went crazy clicking on it. How would your system determine it is an individual just clicking on others ads say 100 times and not be the owners fault?
I've seen where the owners of the sites have no control if someone comes and does this kind of crap. Personally, any clicks should be voided, but in most cases it causes the owners account to be shut down. Are there any systems in place to make this distinguishment and know it is not the owners fault?
We count a person as a unique person. If that person clicks out a million times that is still 1 person. I would never shut somebody down for somebody clicking out a million time, unless I find out it was the owner of the domain doing it.
Donny
Obviously some people may try to use Proxies to fake traffic, so perhaps domainers could also check for this, even if it is to monitor the situation and perhaps filter them out also.
you can check for proxies with this little snippet to get the real IP.
PHP Code:<?function get_ip()
{
if (isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$ip = $_SERVER['HTTP_X_FORWARDED_FOR']; }
elseif (isset($_SERVER['HTTP_CLIENT_IP'])) {
$ip = $_SERVER['HTTP_CLIENT_IP']; }
else {
$ip = $_SERVER['REMOTE_ADDR']; }
return $ip;
}
$ip = get_ip();
?>
Last edited by namestrands; 06-17-2006 at 10:56 AM.
Enterprise Communications and Social Media in Uist
I'm waiting for you to come up with a list so I can ban them.Originally Posted by namestrands
Actually I won't provide a list to the general public, because when I provided it to even our account managers they provide the list to the webmasters. One webmaster who I shut down, I shut down then signed up again with different information, then he just renamed the url he was sending the paid traffic from and changed the domain. But it was the same traffic.
And trust me I have found thousands of domains using the same programs. It just depends on the volume in an account.
Donny
I appreciate that, I will continue to build my list with help from the Domaining community.
I am sure most of us want to rid the world of leechers and such and therefore helping us all with better conversion and transparency.
While we have this we can keep ourselves and of course the parking programs honest ;-)
I am now developing a Parking program status check script that runs every 15 minutes and checks to see if a program is UP or DOWN, in the event of a program not responding it will issue an alert to the users while also triggering a database update to tell the script to redirect all traffic to the second tier program.
Will post once complete.
also noticed a lot of image traffic.. which are of no use to anyone.. however you can filter that out also.
PHP Code:<?
$uri = $_SERVER['REQUEST_URI'];
if ( false !== strpos( $uri, ".gif" ) )
{
header("Location: http://www.domain.com/FATCHICA.GIF");
exit;
}
else if ( false !== strpos( $uri, ".jpg" ) )
{
header("Location: http://www.domain.com/FATCHICA.JPG");
exit;
}
else if ( false !== strpos( $uri, ".png" ) )
{
header("Location: http://www.domain.com/FATCHICA.PNG");
exit;
}
?>
Last edited by namestrands; 06-17-2006 at 11:19 AM. Reason: Automerged Doublepost
Enterprise Communications and Social Media in Uist
The other important thing is to make sure that this is not tracked in your stats. I can see it now, I got 96 hits today and not a single click, duhhh it's your uptime checker...
Donny
lol.. thats a given as it will have its own User Agent, so easily Identifiable and easy to filter from the domainers own stats.
Enterprise Communications and Social Media in Uist
But image traffic is always so much fun! Send me a 404 image and I'll send you a gem.Originally Posted by namestrands
http://phpscripts.com/diamond.gif
I've always wanted to post that.
Donny
yeah but its loaded in an image tag so no use to anyone.. of course unless a link to an actual image was clicked.
hmmm will see what I can do with that
Enterprise Communications and Social Media in Uist
The proxy idea is great. Gets rid of all the illegitimate domainers.
Another idea would be to determine where in the world the traffic is coming from and display ads accordingly. I am sure Parked has advertisers from all of the world, and likely already doing this.
I use geo targetting. I posted a snippet in a previous post about this very thing.
http://www.dnforum.com/post862372-6.html
Enterprise Communications and Social Media in Uist
Bookmarks