What do you think of my new site and idea?

[adill420]

Hello,

Wanted to get an opinion on something I am trying to do.

Briefly I am trying to build a huge archive of all the forums on the internet. To accomplish this I am working on a bot similar to the ones used by yahoo and google.

Would be nice if I can get any advice, suggestions..anything is welcome!

Thank you.

www.ForumArchives.com

*To see a working example of the archive, please click on webmaster directory on the main page and you will see three sites with "View Archive: Click here"

 

[stoned]

Nice idea, Partners?

 

[adill420]

Nice idea, Partners?

Hey,

Thanks. No partners yet. I am not sure if I will be accepting anyone. But then again Not sure yet. Have already put in a lot of work.

Looking forward to get some suggestions.

Thanks

 

[TrafficMonsterRRR]

This is looking great. Good luck indeed. I will add my medical/health forum a bit later on.

 

[adill420]

This is looking great. Good luck indeed. I will add my medical/health forum a bit later on.

Thanks..yes i think it is great as well..just have to figure out how to get traffic..hopefully someone will have some ideas here..

thanks

 

[Steen]

If you continue to archive forums and have people submitting their forums to you, the massive amount of content should bring search engine traffic.

 

[Domagon]

You have a lot of servers available to store all that data and handle the traffic?

An idea to consider is getting voluteers to host portions of your archive ... each on their own subdomain - then tranparently map them all together for the enduser; giving some public credit of course along the way to your volunteers so they stay onboard.

One server / host will quickly be swamped - just to spider sites alone will take a group of servers to perform.

If you are truly committed to archiving forums for research/historical reasons, then register the .ORG version and drop/limit the advertising; keep in mind copyright laws.

Consider starting a non-profit and find others with similar interests with servers/hosting/money who are willing to help ... what you're seeking to do is a absolutely huge task - ever seen Google groups ... now imagine doing that yourself many times over - that's basically what you're trying on an ever larger scale since the collective data size of message boards far exceeds that of the newsgroups Google archives; they don't archive binary groups, but even that isn't much compared to what is out on the web boards now.

If you're seeking to start a business on this, consider getting the permission of a select number of message boards to copy and archive their visitors' messages. This approach would greatly reduce your hardware/hosting needs, reduce legal risks, and more likely to become a viable business venture.

Ron

 

[Crazysah]

That is a good site there.

 

[Edwin]

I assume that you're explicitly getting permission from the forum owners?

I certainly would be VERY unhappy if someone came and copied all my forum data onto their own site, and would probably have to start looking into legal action if they didn't remove it immediately (since the duplicate content might get my own info thrown out of the search engines). Forum posts are protected by copyright just like any other type of content - nothing "magically" allows people to take forum posts for their own use.

 

[fryman04]

Sad but true... just look at all the lawsuits the owner of the wayback site has battled.

 

[cyphix]

I don't think he's planning on archiving the forum contents, rather he is trying to build a bot to find all the forums on the net & link them in his directory.

 

[Honan]

I don't think he's planning on archiving the forum contents, rather he is trying to build a bot to find all the forums on the net & link them in his directory.

Yes he is
e.g.
http://www.forumarchives.com/php-spot/

 

[adill420]

OK, before this gets to any pile of confusion.

I thought about legal aspects and for now this is only for beta testing purpose. But I haven't actually thought about complete setup and process of what to archive.

However, as of now, I will only archive forums whose owners allow and in return (not now, in the near future) along with google adsense and possibly other ad revenue system will recieve a split revenue generated from their archive. So it will be beneficial for the owner of the forum as well. Possible a source of revenue along with traffic.

The main concern for me as of now is not the forum or its owners as I know lots of forums don't have any problems with their data being archived (which technically is same as google spider). The main concern is how to attract users to use the site. What possible features to add. One thing I am considering and possible the main reason for archive is to find information on multi-forums as opposed to just one.

Personally, archiving is not illegal. Which is why I believe robots.txt is the method that a site owner needs to use to block. Archive.org is extremely usefull however I only see purpose for domainers to use it to figure out what traffic existed there before.

If archiving was illegal then Google, yahoo etc are illegal. There is indepth for this, however, again to avoid all the hassle I will only go with consent forum owners to make it easier. Other forums will be listed as a directory no archive.

Thank you all for all the suggestions and heads up. Hopefully some more input will be recieved after this!.

Thanks again!

 

[Edwin]

If you ever change that policy, make sure you:-

A) Provide VERY PROMINENT information about what line needs to be added to robots.txt to block your spider

B) Provide a "remove" mechanism that is fast and thorough to remove data that the owners of the data don't want archived. Archive.org and the Google Cache are two examples of cached data with a clear, publicly displayed process that will lead to the elimination of specific data in a reasonably short timeframe.

The smart forum owners will already have archived their own forums and set up adsense etc. so why would they want to risk having their content thrown out of the search engines for a fraction of ad revenue share when they've already got 100% share themselves?

Personally, archiving is not illegal. Which is why I believe robots.txt is the method that a site owner needs to use to block. Archive.org is extremely usefull however I only see purpose for domainers to use it to figure out what traffic existed there before.

If archiving was illegal then Google, yahoo etc are illegal. There is indepth for this, however, again to avoid all the hassle I will only go with consent forum owners to make it easier. Other forums will be listed as a directory no archive.

On this aspect of your plan, talk to a good lawyer!

It makes NO DIFFERENCE what you THINK is legal, the only thing that matters is what IS legal. A lawyer will be able to tell you exactly where the limits lie.

A few hundred or thousand dollars invested in legal fees up front could help prevent huge and costly exposure later.

Oh, and make sure you have taken out adequate liability insurance.