developed BeTheBot.com - Pretend to be the google bot

[cvxdes]

Hey everyone,
I just finished this site a few days ago, and there's not much to it. I have to remind everyone, this IS NOT a proxy. It just loads a site as if it was it was the google bot, an puts that out. The entire site is custom made, by me.
I'm open to offers. Please post them or PM me the offers

http://www.bethebot.com/

Also comes with bethebot.net

I'll set the BIN at $250, PM me for that.

About 500 uniques today. It's on diggs homepage


Thank you

 

[cshel]

I tried it out and it looks pretty much just like the regular webpage would in a normal browser window. Bots can't "see" pictures... bots don't know what to do with javascript. How precisely is this supposed to "be the bot"?

I'm not trying to be mean, I just want to understand. I like the name and the concept, but I don't think the site actually functions.

 

[cvxdes]

It does function. It took weeks to program

Google "experts exchange javascript sleep", the first result is http://www.experts-exchange.com/Pro...pting/JavaScript/Q_20438455.html?qid=20438455

In the google preview, you can see that someone DID help. You can see part of the solution, but you can't find it when you click the link. That's because they show the results to google, not to you.

Now, go to the site, and paste that same URL. Tada, the answers are at the bottom. Works with tons of other sites.

 

[cshel]

Oh. It doesn't show the page itself as the bot "views" it... it only determines if the site you're trying to access redirects traffic based on whether it thinks you're googlebot or not. So it's just a cloaking/redirection detector, and it's not very clear about what it's supposed to be doing.

It would be more useful and cooler if it displayed the webpage sans images and javascript... the way it looks in Lynx perhaps.

 

[cvxdes]

Oh. It doesn't show the page itself as the bot "views" it... it only determines if the site you're trying to access redirects traffic based on whether it thinks you're googlebot or not. So it's just a cloaking/redirection detector, and it's not very clear about what it's supposed to be doing.

It would be more useful and cooler if it displayed the webpage sans images and javascript... the way it looks in Lynx perhaps.

It shows it as google would so it, as in, it shows what it would hide from you. I think I kind of understand what you're thinking... but that's not what the site does. In the "What is this site?" box, it explains:

Have you ever been googleing something, and you see exactly what you need in the preview, but when you click the link it doesnt show you what you want to see?
This is because the owners of the site are trying to trick you into buying something, or registering. It's a common tacit on the internet. When Google visits the site, it gives something called a "Header". This header tells the site who the visitor is. Google's header is "Googlebot". The programmers of the site check to see if the header says "Googlebot", and if it does, it opens up all of its content for only googles eyes.
Now, all we have to do is trick the site's headers, into thinking that we ARE google. That's what this site does. See the How to use box to the right for instructions on usage

 

[fish]

I'm interested, would like to see a screenshot of the traffic please. Also, where's the domain registered? Thanks.

 

[cvxdes]

BeTheBot.net, #2 story, first page of digg.com

Awesome.