Quote:
|
Originally Posted by DotComGod Seems like it spiders certain engines and I want to build a large search engine.
But I do appreaciate the searching.
Seem like will be a high end commercial package but I cant find it.
-=DCG=- |
How large do you want it? To do a large search engine most likely requires colocated hardware on its own interconnect and lots of RAM, diskspace and bandwidth. The market for such is pretty small, so I am not sure there would be an off the shelf package for it. At least for what I would view as a large search engine. There are some companies that sell the toolkit for making one, but it is far from an off the shelf solution. They also wind up needing tweeking as new ways to spoof and flood it are found, and to improve the ranking relivancy tools, so they are also not really set and forget types of things at this scale. All that being said, I do have a toolkit for making a search engine archived around someplace around here from some work I did like 7 years ago.