Welcome to Welcome to DNF.com™ - Domain Sales, Domain Forum, Domain Appraisals, Domain Registrars

If you are new to domains and looking to buy, sell and learn about domains then you have come to the right place. DNForum is the largest domain name community on the internet and continues to grow every day. There are over 105,000 domainers on DNForum doing everything from buying domains, selling domains, learning about domains and discussing domains. Take a minute and Register.

Register Today on DNForum IT'S FREE!

Results 1 to 10 of 10
  1. 04-06-2009, 05:21 AM #1
    InfoGuy
    InfoGuy is offline
    Platinum Lifetime Member
    Join Date
    Oct 2008
    Location
    Los Angeles
    Posts
    119
    Country

    United States
    DNF$
    332
    Bank
    0
    Total DNF$
    332
    Donate  

    Thumbs down ► Stupid and Abusive Sedo Bot ► sedo_parking_robot

    91.195.241.219 - - [06/Apr/2009:04:33:22 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:22 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:23 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:23 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:23 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:23 -0500] "GET / HTTP/1.1" 200 13897 "-" "sedo_parking_robot"

    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:24 -0500] "GET /index,follow HTTP/1.1" 404 13904 "-" "sedo_parking_robot"

    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /global HTTP/1.1" 404 13904 "-" "sedo_parking_robot"

    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    91.195.241.219 - - [06/Apr/2009:04:33:25 -0500] "GET /text/html;charset=iso-8859-1 HTTP/1.1" 404 13904 "-" "sedo_parking_robot"
    1) Why is sedo's dumb bot hammering my server with multiple requests (6 every time) for the same page from the same IP within 1 second?

    2) sedo's bot is also making numerous unnecessary requests, because it can't differentiate a meta tag from a anchor text link.

    I do not appreciate stupid bots that eat up my bandwidth for no good reason. The crawling patterns of this bot lead me to believe that there is no legitimate reason for it crawling my site, other than to spam me with Sedo's parking service. A Sedo representative is more than welcome to shed some light on this matter.
    Reply With Quote Reply With Quote
  2. 04-06-2009, 05:43 AM #2
    Carter
    Carter is offline
    Success Is My Only Option
    Carter's Avatar
    Join Date
    Jul 2008
    Location
    Italy
    Posts
    4,249
    Country

    Italy
    DNF$
    28,074
    Bank
    0
    Total DNF$
    28,074
    Donate  
    Can we have a sedo Bot Support?
    Reply With Quote Reply With Quote
  3. 04-06-2009, 05:59 AM #3
    mvl
    mvl is offline
    mvl's Avatar
    Join Date
    Sep 2006
    Location
    Netherlands
    Posts
    1,179
    DNF$
    7,326
    Bank
    0
    Total DNF$
    7,326
    Donate  
    What kind of sites does it spider?

    Reply With Quote Reply With Quote
  4. 04-06-2009, 08:51 AM #4
    Acro
    Acro is offline
    Bloody Hell
    Acro's Avatar
    Join Date
    Feb 2004
    Location
    USA
    Posts
    28,665
    Country

    Holy See
    DNF$
    15,555
    Bank
    0
    Total DNF$
    15,555
    Donate  
    Quote Originally Posted by mvl View Post
    What kind of sites does it spider?
    Presumably it checks to see whether the domains are still parked.

    DomainGang.com - Digital Entertainment for Domainers
    Acroplex - Web & Graphics
    Acro.net - My Blog
    Reply With Quote Reply With Quote
  5. 04-07-2009, 09:56 AM #5
    SedoCoUk
    SedoCoUk is offline
    Platinum Lifetime Member
    SedoCoUk's Avatar
    Join Date
    Aug 2002
    Location
    Cambridge, MA
    Posts
    1,146
    DNF$
    3,019
    Bank
    0
    Total DNF$
    3,019
    Donate  
    Quote Originally Posted by InfoGuy View Post
    1) Why is sedo's dumb bot hammering my server with multiple requests (6 every time) for the same page from the same IP within 1 second?

    2) sedo's bot is also making numerous unnecessary requests, because it can't differentiate a meta tag from a anchor text link.

    I do not appreciate stupid bots that eat up my bandwidth for no good reason. The crawling patterns of this bot lead me to believe that there is no legitimate reason for it crawling my site, other than to spam me with Sedo's parking service. A Sedo representative is more than welcome to shed some light on this matter.
    Hello InfoGuy,

    I just sent you a PM for more information.

    Thanks,

    Lisa
    Join Sedo Today!
    Reply With Quote Reply With Quote
  6. 04-07-2009, 10:08 AM #6
    BobDiGiTaL
    BobDiGiTaL is offline
    DNF Addict
    Join Date
    Aug 2005
    Posts
    1,605
    DNF$
    15,064
    Bank
    0
    Total DNF$
    15,064
    Donate  
    Lisa, why so secretive? post your response below.
    BobGuzzo.com domains.
    Reply With Quote Reply With Quote
  7. 04-08-2009, 09:58 AM #7
    SedoCoUk
    SedoCoUk is offline
    Platinum Lifetime Member
    SedoCoUk's Avatar
    Join Date
    Aug 2002
    Location
    Cambridge, MA
    Posts
    1,146
    DNF$
    3,019
    Bank
    0
    Total DNF$
    3,019
    Donate  
    Quote Originally Posted by BobDiGiTaL View Post
    Lisa, why so secretive? post your response below.
    Hello everyone,

    Yes, we do have checks in place to detect domains that are parked with us. I will certainly post a response to this thread, however at this time I am waiting for further information from InfoGuy.

    Lisa
    Join Sedo Today!
    Reply With Quote Reply With Quote
  8. 04-08-2009, 03:20 PM #8
    InfoGuy
    InfoGuy is offline
    Platinum Lifetime Member
    Join Date
    Oct 2008
    Location
    Los Angeles
    Posts
    119
    Country

    United States
    DNF$
    332
    Bank
    0
    Total DNF$
    332
    Donate  
    Lisa,

    My domain is irrelevant. It is the sedo bot that is not behaving properly.

    1) There is no reason for your bot to request the same page 6 times within a 1-second interval. Once every 10 days is considered appropriate for spidering a site.
    2) If the sedo bot is just checking to see if a domain is still parked at Sedo, it should only request Robots.txt and the main index page. There is absolutely no reason for it to deep crawl other pages on a site. Given how poorly Sedo's bot crawls sites, I wouldn't be surprised if it triggered fraudulent clicks on non-Sedo parking pages.

    Nevertheless, I have banned Sedo's bot from my server with .htaccess to prevent future issues.

    To anyone interested in blocking sedo_parking_robot, add the following to your .htaccess file:

    RewriteCond %{REMOTE_ADDR} ^91\.195\.(24[0-1])\. [OR]
    RewriteCond %{HTTP_USER_AGENT} sedo_parking_robot
    RewriteRule .* - [F,L]

    - or -

    SetEnvIf Remote_Addr ^91\.195\.(24[0-1])\. ban

    <Files ~ "^.*$">
    order allow,deny
    allow from all
    deny from env=ban
    </Files>
    Last edited by InfoGuy; 04-08-2009 at 08:39 PM.
    Reply With Quote Reply With Quote
  9. 04-08-2009, 04:21 PM #9
    namenut
    namenut is offline
    Platinum Lifetime Member
    namenut's Avatar
    Join Date
    Apr 2006
    Location
    Las Vegas
    Posts
    205
    Country

    United States
    DNF$
    796
    Bank
    0
    Total DNF$
    796
    Donate  
    Quote Originally Posted by InfoGuy View Post
    Lisa,

    My domain is irrelevant. It is the sedo bot that is not behaving properly.

    1) There is no reason for your bot to request the same page 6 times within a 1-second interval. Once every 10 days is considered appropriate for spidering a site.
    2) If the sedo bot is just checking to see if a domain is still parked at Sedo, it should only request Robots.txt and the main index page. There is absolutely no reason for it to deep crawl other pages on a site. Given how poorly Sedo's bot crawls sites, I wouldn't be surprised if it triggered fraudulent clicks on non-Sedo parking pages.

    Nevertheless, I have banned Sedo's bot from my server with .htaccess to prevent future issues.
    Nicely done InfoGuy .... note how "Lisa" wants to get more info from InfoGuy before she releases any of her own. Obviously, or so seemingly, not wanting shed light on anything more than she/SEDO absolutely has to. Indicative of the services and practices at SEDOn't.



    nn

    A m e r i c a . a m * A p p r a i s a l A p p . c o m * S a f e A p p s . o r g * T e a m U S A . u s
    I really do TRY to keep my mouth SHUT but sometimes I type out loud =0 )
    Reply With Quote Reply With Quote
  10. 04-09-2009, 05:29 PM #10
    SedoCoUk
    SedoCoUk is offline
    Platinum Lifetime Member
    SedoCoUk's Avatar
    Join Date
    Aug 2002
    Location
    Cambridge, MA
    Posts
    1,146
    DNF$
    3,019
    Bank
    0
    Total DNF$
    3,019
    Donate  
    Quote Originally Posted by InfoGuy View Post
    Lisa,

    My domain is irrelevant. It is the sedo bot that is not behaving properly.

    1) There is no reason for your bot to request the same page 6 times within a 1-second interval. Once every 10 days is considered appropriate for spidering a site.
    2) If the sedo bot is just checking to see if a domain is still parked at Sedo, it should only request Robots.txt and the main index page. There is absolutely no reason for it to deep crawl other pages on a site. Given how poorly Sedo's bot crawls sites, I wouldn't be surprised if it triggered fraudulent clicks on non-Sedo parking pages.

    Nevertheless, I have banned Sedo's bot from my server with .htaccess to prevent future issues.
    Hello,

    As I do not have direct access to the information needed to research your issue I spoke with our Technical Team. In order for them to look into this more closely they would need the domain name to review the logs. From your post it was not clear if this involved one domain or several. Unfortunately without this information we cannot identify the issue to provide you with an answer.

    If you would like to PM the domain we can look into this further.


    Thanks,

    Lisa
    Join Sedo Today!
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules

Domain name forum recommended by Domaining.com